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Executive Summary 

Over the last several decades, issues of trust and accountability have moved to the forefront of community-police relations 
and a great deal of scholarship has been devoted to enhancing police performance through the strengthening of law enforce¬ 
ment oversight functions. During this same period, highly publicized officer-involved encounters have led to the proliferation 
of organizational mechanisms for reviewing and improving officer conduct (Walker 2001; Alpert et al. 2016). 

One such mechanism is the development of civilian oversight of law enforcement. Sometimes referred to as citizen oversight, 
civilian review, external review and citizen review boards (Walker 2001; Alpert et al. 2016), this form of police accountability is 
often focused on allowing non-police actors to provide input into the police department’s operations, often with a focus on the 
citizen complaint process. In some jurisdictions, this is sometimes accomplished by allowing oversight practitioners (both paid 
and volunteer) to review, audit or monitor complaint investigations that were conducted by police internal affairs investigators. 
In other jurisdictions, it is done by allowing civilians to conduct independent investigations of allegations of misconduct lodged 
against sworn law enforcement officers. It can also be accomplished through the creation of mechanisms that are authorized to 
review and comment on police policies, practices, training and systemic conduct. Some oversight mechanisms involve a combi¬ 
nation of systemic analysis and complaint handling or review. 

The goal of this publication is to offer a broad examination of the key issues facing civilian oversight of law enforcement in the 
US. Drawing from a review of the available research, as well as organizational data collected from 97 police oversight executives, 
this report is designed to help local policy-makers, police executives and members of the local community explore the key issues 
that can accompany the implementation and sustainability of civilian oversight of police mechanisms at the municipal and 
county levels. More specihcally, this report addresses a number of key areas in relation to police oversight, including: 

1. A comprehensive review of the key resources and research relating to civilian oversight of police; 

2. A brief review of the historical evolution of oversight in the U.S.; 

3. A detailed examination of three different models of oversight: investigation-focused, review-focused and 
auditor/monitor-focused; 

4. An assessment of the key factors that promote organizational effectiveness in civilian oversight; and, 

5. An exploration of trending issues in relation to oversight, particularly the debate over how to measure performance of 
police oversight agencies, the potential value of problem-solving methodologies and the increasing emphasis on the value 
of alternative dispute resolution techniques for resolving complaints against police officers. 

This report concludes by identifying issues that jurisdictions may want to consider if they are evaluating whether to implement 
oversight or revise their current oversight framework. This report also identihes critical areas in need of further research. 

Research and Resources Related to Civilian Oversight of Police 

A reasonably large body of academic and policy literature on civilian oversight exists that dates back to the 1970s. For juris¬ 
dictions that are considering implementing police oversight or are considering reforming the current oversight functions in 
their jurisdiction, there are a number of key publications that can be consulted. For example, the following reports and books 
provide general guidance on the issues that can accompany the implementation of civilian oversight of police: 

• Civilian Oversight of Police: Advancing Accountability in Law Enforcement (Prenzler and den Heyer 2016) 

• The New World of Police Accountability (Walker and Archbold 2014) 

• Citizen Oversight of Law Enforcement Agencies (Perino 2006) 
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• Citizen Review of Police: Approaches and Implementation (Finn 2001) 

• Police Accountability: The Role of Citizen Oversight (Walker 2001) 

Recent publications have begun to examine the broad organizational characteristics of different models of civilian oversight 
in the U.S. They include: 

• Citizen Oversight in the United States and Canada: Applying Outcome Measures and Evidence-Based Concepts (Alpert et al. 2016) 

• Overview of Civilian Oversight of Law Enforcement in the United States (Attard and Olson 2013) 

• Civilian Oversight of Policing: Lessons from the Literature (Miller 2002) 

• Models of Oversight: A Critique (Prenzler and Ronken 2001) 

A limited number of publications, listed below, focus on effectiveness and performance measurement in civilian oversight of 
law enforcement: 

• Holding Police Accountability Theory to Account (Flarris 2012) 

• Performance Indicators for Police Oversight Agencies (Prenzler and Lewis 2005) 

• Evaluating the Performance of External Oversight Bodies (Brereton 2000) 

• The Use of Performance Measurement in Civilian Oversight in the United States (Mohr 2007) 

Several websites operated by non-profit organizations contain useful resources, links to police oversight agencies, historical 
blue ribbon reports and other types of toolkits that can serve as a starting point for jurisdictions considering implementing 
police oversight or revising their current oversight framework. 

• National Association for Civilian Oversight of Law Enforcement (NACOLE), www.nacole.org 

• Police Assessment Resource Center (PARC), www.parc.info 

• Center for Policing Equity, www.policingequity.org 

Brief History of Oversight in the U.S. 

Samuel Walker (2001; 2006), Merrick Bobb (2003) and Alpert et al. (2016) provide detailed historical descriptions of the 
evolution of civilian oversight in the U.S. The history of civilian oversight in the U.S. can be broken down into several distinct 
developmental waves. 

Early Police Commissions, 1880s-1920s. Early forerunners of modern police oversight initially took the form of police 
commissions, which were implemented by Progressive Era reformers in the late 19th and early 20th centuries as a strategy 
to break the hold of political machines on local policing (PARC 2005; Walker 1977; Walker 2001). Most early commissions 
failed to provide meaningful oversight of local police departments, partly due to the fact that early commission members 
were political appointees, had little expertise in policing and tended to become highly deferential to police executives (Walker 
2001; Bobb 2003: 4; Attard and Olson 2013). 

Early Efforts at Establishing Modern Civilian Review, 1920s-1960s. The first modern forms of civilian oversight in 
the U.S. began in several large cities in the middle of the 20th century (e.g, Washington, D.C., Philadelphia and New York City). 
Early efforts often developed out of conflicts between police and local communities of color and generally took the form of 
under-resourced review boards that played a role in receiving complaints and reviewing completed internal police investigations 
into allegations of misconduct filed against officers (Hudson 1971; Terrill 1988; Walker 2001; Alpert et al. 2016). All of the early 
review boards were ultimately abolished in the face of limited resources and fierce resistance from police unions and local pol¬ 
iticians (Bayley 1991; Walker 2001; Walker 2006). 
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Emergence of Investigative Models of Civilian Oversight, 1970s-1980s. Though all of the initial review boards 
failed, another wave of development began in the late 1960s and carried through to the 1980s (Walker 2001; Walker 2006; 
Alpert et al. 2016). This wave has its roots in the Civil Rights Movement and developed, in large part, as a result of pushback 
and protests over police treatment of African-Americans. While the hrst modern oversight agencies were structured as civil¬ 
ian review boards, this wave saw the development of several full investigative oversight agencies that had enhanced resourc¬ 
es, greater durability and expanded organizational authority. Examples of full investigative agencies in this wave included 
the Public Review Commission in Berkeley, CA and the OfRce of Citizen Complaints in San Francisco, CA. 

Emergence of Auditor/monitor and Hybrid Models of Civilian Oversight, 1990s-Present. Another wave de¬ 
velopment began in the 1990s and continues to present day (Walker 2001; 2006; Alpert et al. 2016). During this period, there 
was a rapid expansion of the number of police oversight agencies subsequent to the March 1991 videotaped beating of 
Rodney King by the Los Angeles Police Department (LAPD). While review-focused agencies remained the most common 
form of oversight, this period also saw the development of the first police auditors, including the San Jose Police Auditor and 
the Special Counsel for the Los Angeles County Board of Supervisors in 1993. Hybrid form of oversight agencies, where 
elements of different models of oversight are purposely combined in a single agency, increased during this period. While no 
one measured the exact numerical increase in oversight agencies in the 1990s, it is clear the number grew from a handful in 
the early 1990s to over 100 in 2001 (Walker 2001).' The research team for this report was able to identify 144 oversight 
agencies operating at the local level in the U.S. as of March 2016. 

Contemporary Models of Civilian Oversight 

Over the years, there have been multiple attempts to create schemas to classify civilian oversight of law enforcement programs. 
One of the challenges in accomplishing this feat is that it almost appears that no two civilian oversight agencies in the U.S. 
are identical. Each jurisdiction has its own political, social and cultural tensions that influence the development of each 
oversight entity’s legal authority and organizational structure. 

In this publication, oversight agencies in the U.S. have been grouped into one of three oversight categories based on input 
they provided in conjunction with an evaluation of their public mission statement and foundational legal authority. The three 
categories include: 

Investigation-focused Models. A form of oversight that operates separately from the local police or sheriff s department. 
While the structure, resources and authority of these types of agencies can vary among jurisdictions, these agencies are tied 
together by their ability to conduct independent investigations of allegations of misconduct against police officers. 

Reviezv-focused Models. A type of oversight that focuses its work on reviewing the quality of completed internal affairs 
investigations. Many review agencies take the form of volunteer review boards or commissions and are designed around the 
goal of providing community input into the internal investigations process. Instead of conducting independent investigations, 
review agencies may evaluate completed internal affairs investigations, hear appeals, hold public forums, make recommen¬ 
dations for further investigation and conduct community outreach. 

Auditor/monitor-focused Agencies. One of the newest forms of poKce oversight. While there can be variation in the orga¬ 
nization structure of this type of civikan oversight, auditor/monitor agencies tend to focus on promoting large-scale, systemic reform 
of police organizations while often also monitoring or reviewing individual critical incident or complaint investigations. 


1 Due to the municipal nature of these agencies, it is difficult, f not impossible to identiff all such agencies throughout the U.S. Even with the advent of modern irformation technolog)}, it 

can be difficult to identify oversight agencies when they have no web presence. 
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Organizational Structure and Effectiveness in Citizen Oversight 

To examine variation in civilian oversight, the research team collected organizational data from 97 civilian oversight agencies 
in the U.S. The goal was threefold: 1) to identify national patterns in oversight; 2) to compare organizational characteristics across 
common oversight models; and 3) to examine effectiveness in oversight (See Appendix A for a description of the research 
methods). 

Common Goals of Civilian Oversight of Police. An important issue for any municipality considering implementing 
or revising an oversight agency is to determine the goals for that agency. Based on the organizational data collected, the most 
commonly shared goals across all models of oversight are; 

1. Improving public trust; 

2. Ensuring accessible complaint processes; 

3. Promoting thorough, fair investigations; 

4. Increasing transparency; and, 

5. Deterring police misconduct. 

Even though those hve goals were common to almost all oversight agencies that submitted organizational data, differences 
between oversight models emerged in relation to other types of organizational goals. Auditor/monitor agencies tended to 
rate goals associated with policy review, auditing, statistical pattern analysis and issuing recommendations as critical or im¬ 
portant for their agency. Investigation-focused agencies were more likely than other types of agencies to indicate that improv¬ 
ing accessibility of the complaint process, ensuring thorough investigations, complainant satisfaction and ensuring that 
complainants received justice were critical or important for their agency. Review-focused agencies tended to identify the 
importance of goals associated with the complaint investigation process, but were less likely to rate goals associated with the 
complaint investigation process as critical or important. 

Triggers for Police Oversight in the U.S. Much of the writing on civilian oversight indicates that it tends to emerge out 
of local crises involving local police and community complaints over racially disparate policing and excessive uses of force 
(Walker 2001). The data supplied support that view. The most common trigger involved force-related incidents (49 percent). 
Just under 30 percent of the agencies indicated that racially biased policing played a role in the development of oversight. 
Notably, only 11 percent of the agencies indicated that they implemented their oversight agency as a proactive or preventive 
policy effort and not as the result of a specihc critical incident. 

Age of Oversight Agencies. The 97 civilian oversight agencies reported how long their agency has been in operation. The 
largest proportion of respondents (55 percent) indicated that their agency has been in continuous operation for 16 or more 
years, which is not surprising since the largest increase in the number of oversight agencies occurred in the 1990s. This pattern 
suggests that the field of oversight has begun to mature and has reached a level of some stability. 

Types of Law Enforcement Agencies Overseen. A large proportion of the oversight agencies that provided data indicat¬ 
ed that they were responsible for overseeing municipal police departments (82 percent). The second largest area of jurisdiction 
was for county sherifis (15 percent). 

Size of Law Enforcement Agencies Overseen. The largest proportion of oversight programs oversaw law enforce¬ 
ment agencies (45 percent) that had fewer than 500 sworn employees. However, there was clear variation across the three 
models of oversight. Review agencies were much more likely to report overseeing smaller law enforcement agencies (500 
or fewer officers), while investigative agencies and auditor/monitor agencies were more commonly found in jurisdictions 
with larger law enforcement agencies. 
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Core Elements of Effective Civilian Oversight. Overall, little research has been devoted to examining factors that 
influence the effectiveness of civilian oversight on police. More critically, relatively little is known as to whether different types 
of jurisdiction or authority have a greater or lesser impact on patterns in officer misconduct or organizational reform. Even 
though there is relatively little social science research on this issue, this report draws from an existing body of policy writing 
(see Walker 2003) and organizational data collected from 97 programs to examine 12 related principles that may strengthen 
civilian oversight programs. The principles reviewed include: 

1. Independence; 

2. Adequate j urisdictional authority; 

3. Unfettered access to records; 

4. Full cooperation; 

5. Access to law enforcement executives and internal affairs staff; 

6. Support of process stakeholders; 

7. Adequate resources; 

8. Public reporting/transparency; 

9. Use of statistical pattern analysis; 

10. Community outreach; 

11. Community involvement; and 

12. Respect for conhdentiality. 

Trending Issues in Civilian Oversight of Law Enforcement 

This report explores some of the key recent developments in civilian oversight of police. 

Using Performance Metrics to Capture Oversight Effectiveness. Even though there is a growing need for civilian 
oversight to demonstrate its organizational value, there is currently no consensus on how to measure organizational performance 
in the held of civilian oversight. Even so, a number of studies have sought to explore the value of different performance measures 
(Brereton 2000; Walker 2001; Prenzler and Lewis 2005; Mohr 2007; FUstad and Gottschalk 2011; FauU 2013). In general, a 
number of issues appear clear. First, “sustain rates” cannot be used to reliably compare performance across civilian oversight 
agencies (Walker 2001). Second, there is a growing consensus that other measures can be useful for capturing workload and 
organization performance in oversight. These measures include: 

1. Trends in reported police misconduct (e.g, number of cases hied); 

2. Workload measures (e.g, number of open cases, cases reviewed, number of investigations conducted, number of 
appeals, identihcation of case dehciencies); 

3. Patterns in hndings and outcomes on case investigations; 

4. Timeliness and process efficiency measures (e.g, number of cases assessed in a given time period); 

5. Public opinion survey measures (e.g, satisfaction rates for complainants and officers, community conhdence in the 
police); 


Civilian Oversight of Law Enforcement | Assessing the Evidence 


6. Review and recommendation effectiveness (e.g., number and type of recommendations made, number and type 
implemented); and 

7. Accessibility measures (e.g, accessibility of the office to the public; accessibility of the complaint process overall) (see 
Prenzler and Lewis 2005: 78-82). 

Adapting Problem-Oriented Approaches to Civilian Oversight. Problem-oriented policing has been an important 
approach to law enforcement for more than 30 years. In this approach, patterns of crime and disorder are analyzed careful¬ 
ly to develop new programs that can reduce the incidence of the identified crime (Porter 2016). Even though this strategy 
has been popular with law enforcement, it has not been widely adopted for use in civilian oversight of law enforcement. 
Adapting this approach for use in civilian oversight has the potential to shift the focus of civilian oversight away from a re¬ 
active focus on investigating officers who may have engaged in misconduct toward a proactive, preventive focus that seeks to 
explore patterns in problems and identify solutions than will prevent misconduct from occurring in the future (Walker 2001; 
Harris 2012; Porter 2016). 

Expanding Alternative Dispute Resolution. The use of mediation by civilian oversight to resolve police complaints 
has become an increasingly popular strategy over the last 15 years. Altogether, 29 agencies reported that they had a mediation 
program, up from 20 agencies in 2008 (PARC 2008). There is a growing body of research that supports the argument that 
mediation can have positive benehts for complainants and officers, especially in relation to complaints alleging less serious 
misconduct (Walker et al. 2002; PARC 2008; 45; Schaible et al. 2013). 

Conclusions 

Given that there has been relatively little empirical research on civilian oversight, this project drew strongly from organiza¬ 
tional data collected from 97 civilian oversight programs to identify important issues within the held. Using those data, this 
paper catalogs a number of key issues that jurisdictions may consider when evaluating whether to implement oversight or 
revise their current oversight framework. 

Oversight is common in U.S with at least 144 oversight agencies and almost all large cities having one or more forms of ci¬ 
vilian oversight of police. These patterns, when taken as a whole, seem to indicate a held that is approaching maturity. Large 
jurisdictions that do not have oversight now stand as oudiers, rather than representing the norm. 

Oversight is a complex organizational held. Review-focused agencies were the most common in the data collected from the 
program sites (40 percent), but auditor/monitor and investigation-focused agencies are now common in the held as well (25 
percent and 35 percent, respectively). More importantly, there is an increasing blurring of the boundaries between different 
models of oversight. Even though the oversight models tended to have clear organizational foci, there was substantial cross¬ 
over between models. For example, a majority of auditor/monitor agencies (61 percent) reported the ability to conduct in¬ 
dependent investigations.^ Roughly 43 percent of review-focused agencies and 50 percent of invesrigative agencies reported 
having the authority to audit complaint invesrigarions by the police. Taken as a whole, this indicates that jurisdictions are 
mixing-and-matching forms of oversight authority to meet the needs of their local environment (McDevitt et al. 2005; 11). 

There is clear variation in the budgets among oversight types. In general, the data provided by the 97 civilian oversight agen¬ 
cies demonstrates that some models of oversight have larger budgets than others. Full investigative agencies tended to report 
the largest budgets, followed by auditor/monitor and review agencies. Similar patterns were evident in terms of staffing. 
Investigative agencies reported having the largest number of full-time, paid staff while review-focused agencies were the least 
likely to report having full-time paid staff. This is likely due in part to variation in the size of jurisdictions and the different 

2 Olson and Attard (2016), in a review of oversight models in California, found that auditors were often granted the legal authority to conduct independent investigations, but seldom 

exercised that authority. 
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mandates of the models. Investigative and auditor/monitor agencies were much more likely to be found in big cities/counties 
with large law enforcement agencies. 

Some models of oversight may be more effective at getting recommendations implemented. Almost ail of the oversight agencies 
reported that police executives listened carefully to the recommendations made by oversight staff (78 percent). However, auditor/ 
monitor agencies were much more likely to report that police or sheriff s agencies implemented their recommendations frequent¬ 
ly or very frequently (72 percent) as compared to investigative (42 percent) and review agencies (34 percent). 

Recommendations 

The following recommendations are designed to highlight additional critical issues for implementing oversight and identify 
ways to expand what is known about civilian oversight of law enforcement. 

Recommendations forJurisdictions Considering Whether to Implement or Revise Civilian Oversight 

Jurisdictions shouldfocus on the “Best-Fit” rather than “Best Practice” when considering how to structure 
civilian oversight. A key lesson that can be learned from the literature and the organizational data is that there is not 
necessarily any “best practice” in the creation of a civilian oversight of law enforcement program. Local jurisdictions vary 
across a range of areas and it is unlikely that any single model will work well for all jurisdictions. Instead, jurisdictions should 
focus on specifying the key goals for their oversight program and then identify the “best htting” model of oversight. 

Oversight should employ the least force necessary to accomplish its goals. It can be argued that “the least intru¬ 
sive means of oversight” (Bobb 2003) necessary to achieve police accountability is the best means of approaching the oversight 
function long term. Just as the police are expected to use only that amount of force that is proportionate, necessary and 
reasonable to accomplish their task, so it can be argued that jurisdictions creating or reforming an oversight function should 
similarly accomplish the feat of ensuring police accountability (Bobb 2003). In other words, a jurisdiction seeking to create 
or update an oversight function should choose the least intrusive model of oversight necessary to accomplish the task. If the 
model chosen does not accomplish that objective, a more aggressive form of oversight would then be required. 

A number of resources are available to jurisdictions considering implementing oversight or reforming their 
current oversight framework. Local jurisdictions who are evaluating whether to implement police oversight should 
consult several relatively recent reports that provide detailed cases studies of existing oversight agencies (see PARC 2005; 
McDevitt et al. 2005; Noe 2013; Olson 2016; PARC 2016). To help local jurisdictions gain access to examples of oversight 
policies, legal language and key organizational documents, the National Association for Civilian Oversight of Law Enforce¬ 
ment (NACOLE) has created a companion website to this report that includes up-to-date prohles for model police oversight 
agencies. This website toolkit can be accessed by visiting: www.nacole.org/agency profiles 

Recommendations for Future Research 

More systematic, comparative research is needed. While some researchers have worked to extend what we know 
about oversight (see Walker and Archbold 2014; Prenzler and den Heyer 2016; Alpert et al. 2016), there are stiU tremendous 
gaps in what we know regarding the effectiveness in civilian oversight. More systematic, empirical research is needed before 
strong conclusions can be drawn about what works and what does not work in relation to civilian oversight of police. 

Additional research is needed on the impact that citizen oversight has on public trust in the police and local 
government. Civilian oversight often emerges as a result of conflict between the police and communities of color and is often 
implemented with the goal of improving community trust and conhdence in the police and local government (Walker 2001:55). 
While some scholars have begun to study the relationship between civilian oversight and public opinion (see TarHng and Dowds 
1997; Worden and McLean 2010; Prenzler 2016), little is known about the impact that the implementation of civilian oversight 
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has on public opinion or whether the presence of civOian oversight can improve public confidence. 

Future comparative evaluations of effectiveness need to be based on primary data collected by outside re¬ 
searchers and not just on public reports produced by oversight agencies. Prenzler and Lewis (2005) and Ferdik 
et ai. (2013) have conducted some of their most important work on oversight performance by reviewing publicly available 
agency reports and documents. Unfortunately, the lack of standardization in reporting and measurement between agencies 
makes it difficult to compare across jurisdictions. In addition, public reports created by oversight agencies may tend to un¬ 
derstate challenges and overstate performance. Future research should adopt mixed-methods methodological approaches 
that combine the use of quantitative outcome measurement, qualitative interviews and community members and officers to 
generate forms of data that allow for comparison across jurisdictions (Ferdik et al. 2013:18). 

More support for independent research on oversight will be necessary before key questions relating to the 
effectiveness of oversight can be answered. Federal and state agencies that support applied research in law enforcement 
should consider developing new and sustained funding streams for research into civilian oversight of police (Buchner 2015). 

More work on standards for oversight is needed. Professional oversight associations, like NACOLE, should continue 
to work collaboratively with researchers and other oversight stakeholders to develop standards and guidelines for oversight 
agency performance and the evaluation of oversight programs. 

Local jurisdictions should be willing to evaluate the effectiveness of their local oversight agencies proactive¬ 
ly and independently. In conducting those evaluations, jurisdictions need to have realistic expectations when it comes to 
the impact civilian oversight may have on local police accountability. Oversight is—and should be—only one component of 
a jurisdiction’s police accountability framework. As such, local jurisdictions should seek to evaluate effectiveness based on the 
factors over which local oversight agencies actually have control. 
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What Do We Know About Civilian Oversight of Law 
Enforeement? Assessing the Evidenee 

Introduction 

Over the last several decades, issues of trust and accountability have moved to the forefront of community-police relations, 
and a great deal of scholarship has been devoted to enhancing police performance through the strengthening of law enforce¬ 
ment oversight functions. During this same period, ofEcer-involved shootings, in-custody deaths and claims of racially and 
ethnically biased police practices has led to the proliferation of organizational mechanisms for reviewing and improving 
officer conduct. 

One such mechanism has been the development of civilian oversight of law enforcement mechanisms. Variously referred 
to as citizen oversight, civilian review, external review and citizen review hoards (Walker 2001; Alpert et al. 2016), this form of police 
accountability is often focused on creating a framework that allows non-police actors to provide input into police depart¬ 
ment operations, with a historical—and often primary—focus on the citizen complaint process. Civilian oversight may be 
dehned as one or more individuals outside the sworn chain of command of a police department whose work focuses on 
holding that department and its officers and employees accountable.^ In some jurisdictions, members of the public review, 
audit or monitor complaint investigations that were conducted by police internal affairs investigators. In other jurisdictions, 
civilians conduct independent investigations of allegations of misconduct lodged against sworn law enforcement officers 
(Walker 2001; Finn 2001). Civilian oversight can also be accomplished through the creation of oversight mechanisms that 
are authorized to review and comment on police policies, practices, training and systemic conduct. Some mechanisms 
involve a combination of systemic analysis and complaint handling or review (Alpert et al. 2016). 

Civilian oversight mechanisms are usually implemented based on the assumption that members of the community do not 
have faith in the ability of a police or sheriffs department to police itself (Walker 2001; Miller 2002; Bobb 2003; Harris 2012). 
When the public believes that officers are not being held accountable for violating the law or department policy, then a con¬ 
sensus may develop that misconduct allegations can be more effectively handled by a civilian organization external to the 
police (Bobb 2003; PARC 2005; Alpert et al. 2016). Underlying this view is the belief that having non-police actors play a 
role within the process for handling complaints can lead to more thorough, complete and impartial investigations and hndings 
(Walker 2001; MUler 2002; Harris 2012). A second common assumption is that involving non-sworn individuals in the over¬ 
sight of the police has the potential to increase public conhdence and trust in the police, or at least trust in local government 
more generally (Walker 2001; Harris 2012). 

The implementation of oversight has long been a source of controversy and political conflict. Early on in the development 
of oversight, police unions, police chiefs, local political actors and law enforcement-related professional associations vehe¬ 
mently objected to the implementation of oversight (Gordon-Reed, 1995; Walker, 2006). Although police unions and some 
line officers associations still tend to resist oversight (Tal 2003; Wilson and Buckler 2010), other influential stakeholders, such 
as the International Association of Chiefs of Police, have accepted the idea that civilian oversight can play a crucial role in 
policing (Finn 2001; International Association of Chiefs of Police 2000; Farrow and Pham 2003). In some cases, police unions 
have supported what has been perceived to be more professionalized forms of oversight, such as a monitor or auditor model, 

3 Civilian oversight, as it is defined and applied throughout this report, is distinctly differentfrom internalforms of police accountability, such as internal cffairs, risk management executive 

committees, and constitutional policing advisors (NACOLE 2015). It is aho distinctfrom community oversight of the police. Community oversight involves grassroots or non-profit organizations that 
(attempt to) directly monitor, evaluate, or influence police conduct through means such as video recording police activity, submitting public records or Freedom of Irformation Act requests, attending 
public meetings during which police actions are discussed, or other tactics. Community oversight would include, but would not be limited to, groups such as Cop Watch, Cop Block, or, in some cases. 
Black Lives Matter. These groups are neither non-governmental and do not have legal or administrative authority to oversee the police, and do not act on behalf of a local jurisdiction nor do they have 
legal authority over cases they investigate. 
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over the creation of citizen review committees (Prenzler and Ronken 2001; Finn 2001). In Los Angeles, the Los Angeles 
Police Protective League, the union representing the nearly 10,000 sworn LAPD officers, sponsors an annual civilian over- 
sight-specihc training workshop facilitated by staff from the Office of the Inspector General, which provides civilian oversight 
of the LAPD. More importantly, the rapid increase in the number of oversight agencies in the U.S. indicates that police 
unions have been unsuccessful at opposing the spread of oversight. While there were only a handful of civilian oversight 
agencies in the early 1990s (Walker 2001), there are now more than 140 civilian oversight agencies at the county and munic¬ 
ipal level in the U.S. Almost aU large cities and an increasing number of mid-sized cities have implemented some form of 
civilian oversight (Walker 2001; Harris 2012; Alpert et al. 2016). Moreover, this pattern is not just limited to the U.S.; civilian 
oversight is also widespread in Canada (Ferdik et al. 2013.), Australia and New Zealand (den Heyer and Beckley, 2016), Israel, 
the United Kingdom (Seneviratne 2004), the continent of Europe (Smith 2016) and the Caribbean. Oversight has also become 
increasingly common in Asia (NaUa 2016) and some parts of Africa (Berg and HoweU 2016). 

Police oversight has received signihcant media and political attention since 2014. The President’s Task Force on 21st Century Po¬ 
licing (2015) listed civilian oversight as one crucial pillar of reform and argued that, “Some form of civilian oversight of law 
enforcement is important in order to strengthen trust with the community. Every community should dehne the appropriate 
form and structure of civilian oversight to meet the needs of that community.” {President’s Task Force on 21st Century Policing 
2015:26). In response to media reports and public demonstrations that followed the police-related incidents,'^ new forms of 
police oversight are being implemented in places as diverse as Ferguson, MO, New York City, NY and Sonoma County, CA. 
Efforts are also underway across a wide range of cities and counties to implement police oversight mechanisms, including 
Fairfax County, VA, Newark, NJ and Oxnard, CA. 

Not all recent media and political attention devoted to police oversight has been positive; however, and some jurisdictions 
have struggled to improve the effectiveness of their oversight systems. The Chicago Police Accountability Taskforce, which 
was convened following the public release of a dashboard camera video of the police shooting death of Laquan McDonald 
in November 2015, recently released a report that was highly critical of Chicago’s police oversight and accountability frame¬ 
work, arguing that “Chicago’s police accountability system does not work. The system should identify and investigate mis¬ 
conduct and then impose appropriate punishment. But at every step, there are enormous barriers.” (Police Accountability 
Task Force 2016; 63). The Chicago Task Force made a signihcant number of recommendations directed at strengthening 
Chicago’s police oversight system. In response to the public scandal and hnal Task Force report, Chicago Mayor Rahm 
Emmanuel elected to disband Chicago’s full investigative police oversight agency—the Independent Police Review Author¬ 
ity, which was created in 2007—and replace it with a combination of a police inspector general’s office and a community 
safety oversight board. 

In a similar set of circumstances, a 2014 task force review of police accountability in Albuquerque, NM resulted in substan¬ 
tial changes to its system of police oversight. This review, which was triggered by a series of controversial officer-involved 
shootings, resulted in the implementation of a new independent, non-police agency focused on overseeing investigations into 
allegarions of officer misconduct (Ad hoc Task Force on Civilian Police Oversight 2014). 

Even though police oversight is a common outcome from local controversies relating to officer-involved shootings and other 
critical incidents, there has been relatively little systematic, comparative research on the effectiveness of civilian oversight. As 
a result, the goal of this publication is to examine the current state of knowledge regarding civilian oversight of law enforce¬ 
ment in the U.S. This report reviews the research literature on oversight in conjunction with data collected from 97 civilian 
oversight programs to help municipal executives, policy-makers, community leaders and the general public consider the 
current issues facing civilian oversight of the police. 

4 Incidents include the deaths of Michael Brown (who was shot by a police officer in Ferguson, MO), Eric Garner (who died after the application of a choke hold used by a New York City 

police officer) and Andy Lopez (o- 13-year old who was carrying a replica firearm when he was shot by a Sonoma County Sheriffs deputy). 


14 I 


Civilian Oversight of Law Enforcement | Assessing the Evidence 



With that goal in mind, this publication contains six major sections. 

• The hrst section includes a brief discussion of the methodologies used to review the research literature and collect the 
organizational data from the civilian oversight agencies. This section also contains pointers to several key empirical reports 
and research studies on police oversight and offers links to key online resources that may be of use to jurisdictions that 
are evaluating whether to implement civilian oversight or are undergoing the process of reviewing their existing oversight 
framework. 

• The second section examines the history of civilian oversight, with a particular focus on the evolution of the organiza¬ 
tional forms of civilian oversight in the U. S. 

• The third section provides a more detailed review of the different models of oversight, including investigation-focused, 
review-focused and auditing/monitoring-focused models of oversight. 

• The fourth section turns to the organizational data provided by 97 civilian oversight agencies to explore key issues and 
obstacles to effectiveness in police oversight. It draws from the policy literature on oversight to identify the core indicators 
of effectiveness in police accountability. This section then uses the organizational data to explore variation in the presence 
or absence of those effectiveness indicators across different models of civilian oversight. 

• The hfth section of the report explores trending issues in relation to oversight, particularly the debate over how to measure 
performance in police oversight, the potential value of problem-solving methodologies for oversight and the increasing 
emphasis on the value of alternative dispute resolution techniques for resolving complaints against police officers. 

• The sixth and final section concludes the report by offering recommendations intended to identify ways that can expand 
what is known about civilian oversight of police. 

Ultimately, this report offers a broad discussion of the important issues currently facing the field of civilian oversight of law 
enforcement. The review and discussion are intended to provide a broad framework that can help policy-makers, police execu¬ 
tives and local community members consider the benefits and challenges that the implementation or reform of oversight can 
have in their jurisdiction. 

Research Methodology 

To develop this report, the research team^ used two different methodologies. First, the research team conducted a review of 
the academic and policy literatures on civilian oversight of the police. Second, to understand the state of the held better, the 
research team requested organizational data from police oversight executives in the U.S. with the goal of describing variation 
in the organizational structure, authority and practices of civilian oversight.® In this section, the team briehy presents the 
methods used to accomplish these data collection tasks. ^ 

Literature Review 

To examine the literature on civilian oversight of the police, the authors conducted key word searches on the Internet, using 


5 The primary research team was composed of Joseph DeAngelis, Richard Rosenthal, and Brian Buchner. NACOLE staff members Cameron AlcEllhiney and Liana Perez ond 
University of Idaho students Andrea Pikes and Nicole Waldo f provided extremely valuable research assistance. 

6 The research team also conducted a similar survey of civilian oversight agencies in Canada. However, the data collection phase of the Canadian survey was not completed in time to be 
included in this report. 

7 The research team for this project also conducted a web scrape of the substantive website contentfor 144 police oversight agencies in the U.S. For this part of the project, the research team 
collected all available substantial website materialfor the oversight bodies, including mission statements, reports, brochures, ordinance/statute language, complaintforms, and any other textual content. 
The goal of this data collection was to develop documentation to supplement an electronic survey of oversight executives. 
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Google and Google Scholars search engines. The team conducted key word searches in common academic databases, in¬ 
cluding LexisNexis Academic, Griminai Justice Abstracts, WorldGat, Academic Search Premier and JSTOR. The authors 
reviewed reference lists and citations for previous academic papers and policy reports. Finally, the authors also reviewed 
reference lists provided on the websites for non-proht oversight and law enforcement-related professional associations, such 
as the National Association for Givilian Oversight of Law Enforcement (NAGOLE), Police Assessment Resource Genter 
(PARG), GATO Institute and International Association of Ghiefs of Police (lAGP).® 

A reasonably large body of academic and policy writing on civilian oversight dates back to the 1970s. For jurisdictions that 
are considering implementing police oversight or reforming the current oversight functions in their jurisdiction, there are a 
number of key publications that can be consulted. 

Several reports and books provide general guidance on the issues that can accompany the implementation of civilian oversight 
of police. The following book-length publications provide detailed, foundational advice across a range of concerns relating 
to civilian oversight:® 

• Citizen Review of Police: Approaches and Implementation (Finn 2001) 

• Police Accountability: The Role of Citizen Oversight (Walker 2001) 

• Citizen Oversight of Law Enforcement Agencies (Perino 2006) 

• The New World of Police Accountability (Walker and Archbold 2014) 

• Civilian Oversight of Police: Advancing Accountability in Law Enforcement (Prenzler and den Heyer 2016) 

• Common Sense about Police Review (Perez 1994) 

A body of more recent work has begun to examine the broad organizational characteristics of different models of civilian 
oversight in the U.S.: 

• Citizen Oversight in the United States and Canada: Applying Outcome Measures and Evidence-Based Concepts (Alpert et al. 2016) 

• Getting It Right: Building Effective Civilian Review Boards to Oversee Police (Ofer 2016) 

• Overview of Civilian Oversight of Law Enforcement in the United States (Attard and Olson 2013) 

• Civilian Oversight of Policing: Lessons from the Literature (MUler 2002) 

• Models of Oversight: A Critique (Prenzler and Ronken 2001) 

• Building Legitimacy and Public Trust Through Civilian Oversight (NAGOLE 2015) 

A number of jurisdictions have also produced (or contracted for) the development of independent evaluations of local civil¬ 
ian oversight agencies. Several of these reports, listed below, contain useful comparative case studies of police oversight 
agencies in the U.S.: 

• Review of National Police Oversight Models for the Eugene Police Commission (PARG 2005) 

• Enhancing Citizen Participation in the Review of Complaints and Use of Eorce in the Boston Police Department (McDevitt et al. 2005) 


8 The authors also conducted key word searches of newspapers and other print media sources in the LexisNexis Academic databases. However, initial searches generated several thousand 
articles spanning the period between 1990 and 2016. As a result, a review of print media sources is not included in this publication. 

9 The International Association of Chiefs of Police (lACP) also published a guide for the implementation of civilian oversight in 2000 titled: Police Accountability and Citizen Review. 
Although it is brief, it does offer potentially useful advice about issues thatjurisdictions may want to consider when implementing oversight. The guide is also notable because it was designed as a best 
practices guidefor law enforcement chief executivesfaced with the prospect of civilian oversight being implemented in their jurisdiction. 
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• Findings on Models of Civilian Police Oversight Authorities (Noe 2013) 

• Peer Review of the New Orleans Office of Independent Police Monitor 2016) 

• Law Enforcement Oversight: Limited Independence, Authority, and Access to Information Impede Effiectiveness (Anderson et al. 2015) 

• Analysis of Police Oversight Models for the City of Pasadena (Olson and Attard 2016)'“ 

One of the core—and currently unanswered—questions in relation to oversight is the extent to which local jurisdictions can 
measure the effectiveness and impact of police oversight on police reform. Overall, there have been few systematic, social 
scientihc studies focusing on the impact of citizen oversight (Walker 2001; Prenzler and Lewis 2005; Harris 2012). There are 
likely a number of reasons for this. First, it can be difficult for researchers to gain access to data relating to police complaints 
and internal investigations. State law often protects these types of personnel records and law enforcement and police oversight 
agencies are often unwilling (or unable) to grant outside researchers access to these types of records. More importantly, there 
are considerable difficulties in designing research that can parcel out the distinct influence that oversight has on policing and 
police reform. The work of oversight agencies is often intertwined with the work of internal police accountability mechanisms 
(Brereton 2000; Prenzler and Lewis 2005). As such, it can be difficult to establish the direct impact that oversight has had on 
the quality of internal investigations or improvements to police operations, policies or procedures. That said, there are a 
limited number of publications focused on effectiveness and performance measurement in civilian oversight of law enforce¬ 
ment. These include: 

• Holding Police Accountability Theory to Account (Harris 2012) 

• Effectively Implementing Civilian Oversight Boards to Ensure Police Accountability and Strengthen Police-Community Relations (King 2015) 

• Arrested Oversight: A Comparative Analysis and Case Study of How Civilian Oversight of the Police Should Function and How it Fails 
(Clarke 2009) 

• Performance Evaluation of Police Oversight Agencies (Filstad and Gottschalk 2011) 

• Performance Indicators for Police Oversight Agencies (Prenzler and Lewis 2005) 

• Evaluating the Performance of External Oversight Bodies (Brereton 2000) 

• The Use of Peformance Measurement in Civilian Oversight in the United States (Mohr 2007) 

There are currently no agreed upon national standards for police oversight. Even so, there have been several efforts, docu¬ 
mented in the citations below, to memorialize national guidelines for certain types of police oversight, including police audi¬ 
tors and monitors: 

• Core Principles for an Effective Police Auditor’s Office (Walker 2003) 

• National Guidelines for Police Monitors (Bobb et al. 2008) 

Beyond these academic and policy publications on police oversight, a number of websites operated by non-proht organizations 
contain useful resources, links to police oversight agencies and other types of toolkits that can serve as a starting point for 
jurisdictions considering implementing police oversight or revising their current oversight framework: 

• NACOLE, www.nacole.org 

• Police Assessment Resource Center (PARC), www.parc.info 

• Center for Policing Equity, www.policingequity.org 

10 Olson (2016) also recently conducted a review of key issues confronting the Spokane County Sheriffs Citizens Advisory/Review Board, which peforms some oversightfunctions for the 

Spokane County Washington Sherffs Department. This report may be of particular interest to jurisdictions that are thinking about transitioning a purely internal advisory board into a broaderform of 
independent, external citizen oversight. 
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Electronic Organizational Data Collection 

Since much of the recent research on civilian oversight has been based on either case studies or anecdotal accounts of indi¬ 
vidual agency practices, there are substantial gaps in what is known about the state of civilian oversight in the 21 st century. 
To develop better understanding of the general state of civilian oversight in the U.S., the research team requested organiza¬ 
tional data from 97 civilian oversight agencies in the U.S. The overall goal was to capture and describe variation in the or¬ 
ganizational structure, legal authority, organizational practices and hnancial resources for contemporary oversight. Several 
of the attitudinal questions were adapted from early survey research on oversight (Walker and Bumphus 1991). The research 
team designed the data request to measure organizational and attitudinal information in 12 key areas: 

1. The history of the oversight agency; 

2. The legal authority/organizational powers granted to the agency; 

3. Agency goals; 

4. Organizational structure; 

5. Budget; 

6. Staffing; 

7. Presence and structure of a board or community members; 

8. Role in the complaint intake and investigation process; 

9. Community outreach practices; 

10. Perceived support from process stakeholders; 

11. Data collection and analysis practices; and 

12. Key successes or obstacles that the agency has experienced in the previous three years. 

See Appendix A for further details about the methodology. 

Brief History of Civilian Oversight 

Early forerunners of modern police oversight initially took the form of police commissions, which were implemented by 
Progressive Era reformers in the late 19th and early 20th centuries as a strategy to break the hold of political machines on 
local policing (PARC 2005; Walker 1977; Walker 2001; Walker 2006). To reduce the influence of partisan politics and nep¬ 
otism on police departments, these commissions were populated by part-time, non-police volunteers and often granted the 
power to hire and hre the local police chief The goal was to support the development of a professionalized, independent 
police force by having the commission function as a buffer between local political actors and the police chief (Walker 2001; 
Walker 2006). Unfortunately, early commissions never functioned as effectively as initially envisioned, partly due to the fact 
that commission members were political appointees and had little expertise in policing (Walker 2001; Bobb 2003). As a result, 
they tended to become highly deferential to police executives and failed to provide meaningful oversight of local police de¬ 
partments (Bobb 2003: 4; Attard and Olson 2013).*' Later efforts to establish police oversight arose out of the Civil Rights 

11 A number of police commissions remain in existence today, such as DetroiPs Board of Police Commissioners, the Kansas City Board of Police Cojnmissioners, the Honolulu Police 

Commission and the Los Angeles Board of Police Commissioners. These modern examples of police commissions perform some of the same functions as Progressive Era commissions (e.g, commission¬ 
ers are typically volunteers and they hire/fire the police chief. However, they have also evolved considerably from early commissions, operating with greater independence, large budgets, full-time 
professional staff and playing a significant role in overseeing police policy, complaint investigation and disciplinary processes. 
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Movement, in direct response to the treatment of the black community by the police (Walker 2001). The intersection of race 
and policing is central to the history of civilian oversight of law enforcement in the United States; issues of race, equality, 
justice and trust in law enforcement have had a prominent role in police reform over the last 50 years. 

Early Efforts at Establishing Civilian Review, 1920s-1960s 

Support for the implementation of civilian oversight can be seen as early as the 1920s, with most of the efforts focused on 
creating bodies that could better respond to complaints against the police. In 1928, the Los Angeles Bar Association created 
a non-governmental committee to record complaints against police officers (Walker 2006: 3; Ferdik et al. 2013). More 
substantive proposals to create independent, governmental mechanisms for handling complaints against police officers 
can be seen in a series of national commission reports in the 20th century.'^ In 1931, the Wickersham Commission Report 
argued that “some disinterested agency” should be created in “every locality” to take community complaints against the 
police (Wickersham Commission 1931(1968): 192; see also Walker 2006). In 1968, the Kerner Commission (created as 
the result of race-related rioting in United States cities between 1964 and 1968) recommended the creation of external 
oversight, specihcally, “a specialized agency with adequate funds and staff, [to] be created separate from other municipal 
agencies” (United States Kerner Commission 1968: 312). The Kerner Commission also recognized that complaints could 
have value for organizational policy analysis and that the wider public would be less likely to trust the result of complaint 
investigations if those investigations were handled internally by a police department (p. 312; see also Walker 2006). 

Initial efforts to implement modern forms of civilian oversight began in several large cities in the middle of the 20th century, 
with a focus on creating volunteer review boards that would play a role in receiving complaints and reviewing completed 
internal police investigations into allegations of mis¬ 
conduct filed against officers (Hudson 1971; Terrill 
1988; Walker 2001). For example, a Complaint Review 
Board (CRB) was created in Washington, D.C. in 
1948 as a result of community concern about police 
racism and use of excessive force against Afri¬ 
can-Americans (Beattie and Weitzer 2000). The CRB 
existed for almost hfty years, but was poorly funded 
by the city and unable to keep up with an increasing 
caseload (Beattie and Weitzer 2000). The CRB was 
eventually replaced by the Office of Police Complaints 
(Walker, 2001). In Philadelphia, a Pohce Advisory 
Board was created in 1958, but was shut down in 
1969 due to a lack of funding (Terrill, 1988; Walker 
2001). The New York City Civilian CRB was initially established in 1953 as an internal review body staffed by three Deputy 
Commissioners (Walker 2006). It was subsequently made into an external review body, but was disbanded shortly thereafter 
following a public referendum orchestrated by the Policeman’s Benevolent Association (Hudson 1971). 

Examining the early US. oversight agencies, the research team found several key characteristics common to each of these 
efforts. First, the implementation of each of these oversight mechanisms emerged from local crises and civil unrest relating 
to conflict between the police and local communities of color. Second, these oversight agencies were designed around a ci¬ 
vilian review model—that is, they were largely composed of volunteer members with relatively little expertise in police issues. 


12 There were also a number of proposals offered by non-governmental actors that outlined both the rationale and possible organizationalfeatures of possible independent, non-police 

complaint intake and investigation procedures. See, for example, Beral and Sisk’s (1964) law review note comparing the internal police investigation procedures to potential models of citizen review of 
complaints. 


Key Research on Oversight History 

• The History of Citizen Oversight (Walker 2006) 

• Civilian Oversight of the Police in the United States (Bobb 2003) 

• Civilian Oversight of Police in Philadelphia: The First 50 Tears (Tal 
2003) 

• Race, Democracy and Law: Civilian Review of Police in Washington, 
DC (Beattie and Weitzer 2000) 

• Police Accountability in Philadelphia: Restrospects and Prospects 
(Terrill 1988) 
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had small or non-existent budgets and little staff support (Jones 1994; Walker 2001). Their core function was to receive com¬ 
plaints and review completed internal affairs investigations (Walker 2006) but their extremely limited resources compromised 
their ability to keep up with increasing complaint workloads (Ferdik et al. 2013). Third, these agencies all encountered sig- 
nihcant resistance from police unions, local politicians and policy makers, which ultimately resulted in their abolition (Bayley 
1991; Walker 2001; Walker 2006). 

Emergence of Investigative Models of Civilian Oversight, 1970s-1980s 

Although all of the oversight agencies implemented during the initial wave failed, another wave of development began in 
the late 1960s and carried through to the 1980s (Walker 2001; Walker 2006; Alpert et al. 2016). As with the hrst wave, these 
oversight agencies were primarily concentrated in large cities and their implementation was not the product of slow, delib¬ 
erate policy-making but instead driven by local crises as the result of civil unrest and police-community conflict (Walker 2001). 
However, while the initial wave of oversight agencies were structured around limited review board models, oversight agencies 
implemented in the 1970s and 1980s tended to have enhanced resources, greater durability and expanded organizational 
authority (Walker 2006). Moreover, while only a small number of oversight agencies were created in the 1960s through the 
1980s, most of these agencies have remained operational in the long term. A city ordinance in 1969 created the Kansas City, 
MO Office of Community Complaints (OCC) and it has remained in continuous operation through the present day. Unlike 
earlier board models, the OCC had expanded professional staff support that allowed it to respond more effectively to increas¬ 
ing workload demands. The OCC’s legal authority was also enshrined in ordinance, which served as an obstacle to easy 
repeal (Walker 2001; Ferdik et al. 2013). 

Shortly after the OCC was created, a number of other oversight agencies were also implemented and granted the power to 
conduct investigations that were entirely independent of the police. A city ordinance in Berkeley, CA in 1973 created the 
Police Review Commission (PRC) and granted it the ability to independently investigate complaints hied by members of the 
public against police officers (Walker 2001). Nearly ten years later in 1982, an amendment to the City Charter created the 
Office of Citizen Complaints in San Francisco, CA. The Office of Citizen Complaints completely replaced the police inter¬ 
nal affairs function in relation to citizen complaints and was granted the authority to both receive and investigate all citizen 
complaints (the police department continued to investigate internally-generated complaints against officers) (Walker 2001; 
Ferdik, Tklpert, and Rojek 2013). Finally, an ordinance in 1986 reconstituted the Civilian Complaint Review Board (CCRB) 
in New York City. The CCRB initially employed civilian investigators who worked alongside police employees and were 
supervised by police managers; it became an all-civilian investigative oversight agency in 1993 (Walker 2006). 

Emergence of Auditor/monitor and Hybrid Models of Civilian Oversight, 1990s-Present 

Another wave of development began in the 1990s and continues to present day (Walker 2001; Walker 2006; Alpert et al. 
2016). During this period, there was a rapid expansion of the number of police oversight agencies in the U.S. subsequent to 
the March 1991 videotaped beating of Rodney King by the LAPD and the resulting report published four months thereafter 
by the Independent Commission on the LAPD (“Christopher Commission”). 
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While no one has measured the exact increase in 
the number of oversight agencies since the 1990s, 
it is clear the number increased from a handful 
in the early 1990s to over 100 in 2001 (Walker 
2001).*^ One of the core characteristics of this 
wave was the increasing diversity in the organi¬ 
zational structure and legal powers of the differ¬ 
ent organizational forms of police oversight. 
Moreover, the 1990s saw the emergence of a new 
model of oversight—the auditor/monitor model 
(Bobb 2003; Walker 2006). Unlike earlier models 
of oversight that tended to focus on either re¬ 
viewing or investigating individual complaints, 
these auditor/monitor agencies had the mandate 
to examine systemic patterns in complaints or 
other types of officer conduct (including uses of 
force, pedestrian and traffic stops, arrests and 
detentions), and to offer recommendations for 
improving police policies, practices and training. 
While the implementation of review-focused and 
investigatory oversight agencies remained 
common in the 1990s,''‘ a number of oversight 
entities began to focus increasingly on systemic 
causes of misconduct as opposed to focusing 
solely on outcomes of individual complaint in¬ 
vestigations (Lewis 2000; Livingston 2004; Harris 
2012 ). 

The first auditor-focused oversight agency was 
implemented in 1993 in San Jose, CA and was 
followed a short time later by the Seattle Police 
Auditor (Walker 2006; Ferdik et al. 2013). Simi¬ 
larly, as the result of the response of the City of 
Los Angeles to the videotaped beating of Rodney King, a voter initiative in 1995 created the Office of the Inspector General 
(OIG) for the City of Los Angeles and gave it authority to investigate and audit complaint (and other) investigations conducted 
by the LAPD to ensure that they complied with accepted standards (Office of the Inspector General n.d.). Shortly thereafter, 
and in the wake of the Kolts Report and recommendations, the County of Los Angeles adopted the Special Counsel model 
with jurisdiction over the Los Angeles County Sheriffs Department. The Special Counsel was an independent entity that had 
the ability to audit and investigate any area of the Sheriff s Department’s operations, policies and procedures, as well as to monitor 
the implementation of reforms outlined in the Kolts Report. 


Key Events in the Development of Civilian Oversight 

• 1931; National Commission on Law Observance and Enforce¬ 
ment (Wickersham Commission) Report on “Lawlessness in Law 
Enforcement” 

• 1948; Creation of Washington D.C. Complaint Review Board. 

• 1968; Kerner Commission Report recommending External 
Oversight for Police. 

• 1969; Creation of the Kansas City, MO Office of Community 
Complaints (OCC); the longest continuously operating oversight 
agency in the U.S. 

• 1973; Creation of the hrst Independent Investigations Office in 
Berkeley, CA. 

• 1991; Rodney King beating and Christopher Commission 
Report, LAPD 

• 1993; Implementation of the hrst police auditor program in the 
U.S.—San Jose Independent Police Auditor. 

• 1993; Creation of the hrst monitor program in the U.S. — Special 
Counsel for the Los Angeles County Sheriff Department. 

• 1993; The New York City CCRB becomes completely civUian- 
ized. 

• 1994; Enabling legislation for federal “Pattern and Practice” civil 
lawsuits by the Department of Justice, Civil Rights Division (42 
U.S.C. § 14141) 

• 2001; Over 100 oversight agencies identihed in the U.S. 

2016; 144 oversight organizations identihed in the U.S. 


13 This wave of development was accompanied by the growing recognition that citizen oversight constituted a professional field of work and saw the development of professional associations 

that brought together different oversight agencies from both inside and outside of the United States (Walker 2006). The International Association of Civilian Oversight of Law Enforcement 
(LiCOLE) was created in 1985, NACOLE was created in 1995 and the Canadian Association for Citizen Oversight of Law Enforcement (CACOLE) established in 1997. Websitesfor the last 
two professional associations can be visited at: www.nacole.org and www.cacole.ca. 


14 While police auditors emerged as an important form of oversight during this period, it is important to note that there continued to be a rapid increase in the number of citizen review 

boards, which comprised the most common form of oversight implemented during the late 1990s in the United States. 
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By the late 1990s and early 2000s, the numerical rise in the number of oversight agencies continued. This time period also 
saw two critical developments in independent police oversight. First, the U.S. began to see the development of a second 
generation of hybridized forms of civilian oversight, which often emerged as replacements for earlier civilian review boards. 
For example, Portland, OR implemented the Independent Police Review Division (IPR) in 2001 as a replacement for an 
earlier review board. The IPR was consciously designed to draw its organizational structure from different models of oversight 
(Office of the City Auditor 2001). It was empowered to serve as an initial intake point for community complaints against 
police officers; monitor on-going serious investigations; audit complaint investigations conducted by police internal affairs 
staff; and develop data-driven recommendations for improving police policies, practices and training (Independent Police 
Review Division 2002). A later revision to the IPR’s ordinance also granted the agency the ability to conduct independent 
investigations and ultimately monitor police critical incident investigations (Independent Police Review Division 2012: 1). 
The IPR was also created with a citizen oversight board, the Citizen Review Committee, which had the authority to review 
and audit closed complaints, make policy recommendations, hold public hearings and serve as an appellate body for appeals 
of non-sustained hndings made by the Portland Police Bureau (Independent Police Review Division 2003: ix). Similarly, other 
large cities, including Denver (2005) and New Orleans (2009) implemented hybridized auditor/monitor oversight agencies. 

Contemporary Models of Civilian Oversight 

Over the years, there have been multiple attempts to create schema to classify approaches to civilian oversight of law enforcement. 
One of the challenges in accomplishing this feat is that it appears that almost no two civilian oversight agencies in the U.S. are 
identical. Each jurisdiction has its own political, social and cultural tensions that influenced the development of each oversight 
entity’s legal authority, structure and organizational practices (NACOLE 2015). Oversight agencies may receive, review, audit and 
monitor police investigations into officer misconduct. Oversight agencies may conduct investigations that are wholly independent 
of the police. They may analyze and issue public reports on misconduct investigations, lawsuits, uses of force or detentions and 
arrests. Oversight investigators or auditors may provide on-scene monitoring of critical incidents, such as officer-involved shootings 
or protests; and they may evaluate whether an officer’s actions were appropriate under the circumstances or showed a need for 
individual accountability or systemic reform. In addition, oversight entities may also assess a police department’s policies, supervision 
and management practices and training, hiring and recruitment standards, among other procedures. 

Nonetheless, it is possible to review the previous literature and make several initial observations about contemporary forms 
of civilian oversight. These include: 

• High Variability in Organizational Structure. Currently, there is a tremendous amount of variation in the structure 
of different oversight agencies (Walker and Kreisel 1996; Walker 2001; Bobb 2003; Alpert et al. 2016). Some agencies 
operate almost completely by a small number of community volunteers while others have a large number of paid pro¬ 
fessional staff. Some oversight agencies have no operating budget while other agencies have multimillion-dollar budgets. 

• Wide D^erences in Organizational Authority. Substantial variation exists in the jurisdictional authority granted 
to police oversight agencies. Elements such as the role that oversight agencies play in relation to the intake of complaints, 
the relationship they have to the complaint investigation process and their level of access to police records vary. A long 
list of other characteristics differ greatly including whether they can make recommendations as to hndings and discipline, 
or make recommendations relating to police policies, practices and training (Walker and Kreisel 1996; Walker 2001; 
Bobb 2003; Alpert et al. 2016). 

• Organizational “Hybrids” are Common. While early forms of oversight tended to operate as “citizen review 
boards,” and focused on reviewing and commenting on completed internal affairs investigations, many contemporary 
oversight agencies combine different organizational forms and types of organizational authority in relatively complex 
ways (Walker 2001; Finn 2001; Attard and Olson 2013; Alpert et al. 2016). 
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Even given the variability in the structure of oversight, there have been multiple efforts to classify police oversight agencies based 
on key characteristics. Walker (2001) developed one of the earliest and most sophisticated classification systems for oversight.'’ 
Describing the different models as Class I, Class II, Class III and Class IV systems. Walker argues that models of oversight should 
be considered along a continuum that range from forms of oversight that are the most independent from police departments to 
oversight systems that are the least independent. He defines Class I systems as agencies that are independent of police depart¬ 
ments and conduct fuUy independent investigations into allegations of officer misconduct. Class II systems review and comment 
on internal investigations conducted by the police. Class III systems function as appellate bodies, with complainants filing appeals 
with the oversight agency when they are dissatisfied with the outcomes on complaints investigated by local law enforcement. 
Class IV systems are agencies that have the ability to audit, monitor or review the police/sheriff department’s complaint handling 
system. In addition to these classes. Walker also recognizes “hybrid” oversight agencies that do not fit easily within any of the 
four categories (Walker 2001: 62). 

Since Walker developed this classification scheme in the late 1990s, a number of others attempts to update it have occurred. 
Ferdik, Alpert and Rojek (2013) adapted Walker’s (2001) classification schema to explore organizational variation in U.S. and 
Canadian oversight agencies. Using publicly available reports and documents from several large civilian oversight agencies, 
Ferdik et al. (2013) concluded that there was substantial organizational variation across different oversight agencies and new 
“holistic” forms of oversight increasingly blur the boundaries between traditional models of oversight. They argue that these 
forms of oversight can perform traditional oversight roles (e.g, investigation, auditing, and review), but can also conduct 
proactive pattern analysis to offer recommendations for improving policy, practices, and training. Ferdik at al. (2013) con¬ 
tended that this new proactive policy recommendation ability can be found across three of the four “classes” that Walker 
developed. 

In 2005, PARC conducted a research project for the city of Eugene, OR. In it, they outlined a three-part classification scheme 
(PARC 2005). 

1. Review and Appellate models, which are designed to review completed police internal investigations or hear appeals from 
the public on investigation findings; 

2. Investigative and Quality Assurance models, which replace the pohce internal affairs process in whole or in part; and 

3. Evaluative and Performance-based models, which adopt a holistic approach to evaluating patterns in police risk management, 
performance, operations or other organizational systems to promote systemic reform. 

In another recent review of models of oversight, Attard and Olson (2013) revised Walker’s oversight schema and grouped 
oversight agencies based on their role in the complaint handling process as well as their organizational structure. According¬ 
ly, they grouped oversight agencies into three categories: 

1. Investigative agencies, which conduct independent investigations of complaints filed against police officers; 

2. Auditing/monitoring agencies, which systematically review and examine police internal investigations and other law 
enforcement activity to make recommendations around policy and training; and, 

3. Review boards and commissions, which includes a diverse range of agencies headed by volunteer community members 
who may hold community forums, hear appeals or issue findings on investigations completed by paid staff (Attard 
and Olson 2013: 3-5). 


15 A number of other academics and practitioners have developed classification schema that group police oversight agencies in slightly different ways (Premier and Ronken 2001; PARC 

2005). 
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This report adopts an oversight classihcation scheme that is a slightly revised version of Walker’s (2001). It groups oversight 
agencies into three categories based on the core agency functions: 

1. Investigation-focused, 

2. Review-focused,'® and 

3. Auditor/monitor-focused. 

Grouping agencies into these categories required a two-step process. First, an initial quantitative analysis was conducted using 
the organizational data provided by the oversight agencies. More specihcally, agencies were initially grouped based on data 
provided, which focused on how frequently the oversight agencies conducted independent investigations, audited internal 
affairs investigations, monitored open internal affairs investigations and reviewed completed internal affairs investigations. 
During the second stage of the analysis, the agencies were grouped into different organizational categories after the organi¬ 
zational data provided by the oversight agencies was evaluated against their agency’s public mission statement and founda¬ 
tional legal authority (when available).'^ This two-stage process allowed each agency to be grouped into one of three catego¬ 
ries: auditor/monitor-focused, investigation-focused and review-focused. 


Table 1: Classification Schema for Agencies Providing Data for this Report 



Frequency 

Percent 

Auditor/monitor-focused 

24 

24.7% 

Investigation-focused 

34 

35.1% 

Review-focused 

39 

40.2% 

Total 

97 

100% 


In the remainder of this section, the key characteristics of the three different models of oversight are examined. 

Investigation-Focused Models 


One relatively common model of oversight is the investigation-focused agency that operates separately from the local police 
or sheriff’s department (Walker 2001: 62). While the structure, resources and authority of these types of agencies can vary 
among jurisdictions, these agencies are tied together by their ability to conduct independent investigations of allegations of 
misconduct against police officers (Prenzler and Ronken 2001; Walker 2001). These oversight agencies may either complete¬ 
ly replace the police internal affairs function or they may conduct investigations that parallel or duplicate the work of inter¬ 
nal affairs (Finn 2001; PARC 2005). As one example, San Francisco’s Office of Citizen Complaints (OCC) is an entirely ci¬ 
vilian, governmental agency solely responsible for investigating complaints filed by community members against sworn 
members of the San Francisco Police Department (OCC 2016). 

The organizational structure of investigative agencies can vary significantly. In some cases, an investigative agency may be 
governed by a volunteer board (and supported by a professional staff of investigators). In small jurisdictions, an investigative 
agency may be staffed by a single investigator or consultant (Finn 2001; PARC 2005). 

The available literature on full investigative oversight agencies identifies a common set of organizational functions (Walker 


16 Unlike WalkeUs oversight classification scheme, this publication did not include appellate agencies (Class III gystems) as a separate category. Patterns in the organizational data indicated 
appellate Junctions can be found across a range of agencies, including investigative and auditor/monitor ystems that operate with a combination of professional staff and oversight boards composed of 
community volunteers. 

17 Auditor/monitor agencies were collapsed into one category due to significant overlap between agencies that reported performing both auditing and monitoringfunctions either “all the time^’ 
or “very frequently. 
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and Kreisel 1996; Prenzler and Ronken 2001; Walker 2001; Finn 2001; Bobb 2003; Attard and Olson 2013; Ferdik et al. 
2013; Walker and Archbold 2014). Investigative agencies may: 

• Serve as the intake point for public complaints against police officers (Bobb 2003). 

• Review and classify the nature of the complainants’ allegations (King 2015). 

• Conduct independent interviews of complainants, officers and witnesses (Attard and Olson 2013). 

• Be staffed by non-police “civilian” investigators, although some agencies may employ retired or former police officers 
(Finn 2001).'« 

• Be led by a community board or commission that may hold hearings, issue subpoenas or make hndings on investiga¬ 
tions conducted by professional non-police investigative staff (Attard and Olson 2013). 

Potential Strengths of Investigation-focused Agencies 

An investigation-focused agency with appropriately trained staffcan complete thorough and impartial investigations (Prenzler 
and Ronken 2001; PARC 2005). Investigation-focused agencies are the most independent forms of oversight (Walker 2001) 
and tend to have more resources and larger staffs than other types of oversight. Their investigators are also likely to have had 
highly specialized training and experience in relation to investigations, particularly as the organization matures. Thus, where 
investigation-focused agencies are sufficiently resourced, have well-trained, competent staff and are granted sufficient access 
to department personnel and records, they may be able to improve the quality of internal investigations. Even though this is 
a commonly identihed strength of the investigation-focused oversight agency, more rigorous comparative research is needed 
on this issue. 

A related potential strength of the investigation-focused model is its ability to increase public faith in the integrity of the inves¬ 
tigations process, especially in the aftermath of signihcant public scandals involving the police. Available public opinion research 
demonstrates strong public support for the independent investigation of serious complaints against police officers (Prenzler 2016). 
Most investigation-focused agencies utilize civilian staff to conduct fact-hnding investigations and operate a multi-member com¬ 
munity board that may hold hearings, issue findings and/or make recommendations to the police department. As a result, this 
model may reassure a community that investigations are unbiased, thorough and that civilian perspectives are represented both 
within the complaint investigation process and upon review of completed investigations (PARC 2005). 

Finally, relative to other forms of oversight that rely on community volunteers, investigative agencies are able to hire full-time in¬ 
vestigators who often have highly specialized training, greater resources and more time to perform their oversight work (Finn 2001). 

Potential Model Limitations 

Another potential limitation of the investigative model is the signihcant costs and resources necessary to conduct competent, 
timely investigations, including large staffing requirements, as well as the complex organizational issues that can accompany 
the implementation of a stand-alone investigative oversight agency. Full investigative agencies are more expensive than other 
models of oversight, largely due to the increased personnel costs that accompany the hiring of professional investigators (Finn 
2001; vii). 

While the cost of an investigation-focused oversight agency is, by necessity, higher than the other models of oversight, the 
higher cost could be mitigated by the savings realized from a reduction or the elimination of personnel needed to conduct 
police internal investigations. For example, evaluation research conducted in the 1990s on the now-defunct Minneapolis 


18 Some Canadian independent investigation agencies employ “seconded^’ officers who are currently serving police officers assi^ed as full-time investigators serving at the pleasure of the 

oversight agency director, (e.g, the Alberta Serious Incident Response Team (ASIRT) and the Nova Scotia Serious Incident Response Team (SiRT)). 
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Civilian Police Review Authority (CRA) found that CRA investigations were far less costly to conduct on a per case basis 
than investigations conducted by the police internal affairs unit ($3,649 per CRA investigation versus $6,278 per police in¬ 
ternal affairs investigation). This was due, in part, to the lower costs associated with using volunteers and civilian investigators 
(Minneapolis Civilian Review Authority 1997: 18-17; 

Walker 2001; 137). 

Another potential weakness is that attempts to implement 
investigation-focused agencies tend to generate significant 
resistance from police unions and their allies (King 2015). 

Unions have routinely argued that civilian investigators 
do not have the technical background or professional 
experience to conduct complex investigations into allega¬ 
tions of officer misconduct (Prenzler and Ronken 2001; 

Walker 2001). Police unions have also opposed the imple¬ 
mentation of full investigatory oversight agencies, arguing 
that they will be biased against police officers (King 2015). 

In addition, it can be argued that the use of former police 
officers or even civilian investigators who have not pre¬ 
viously served as police officers may not eliminate pro-po¬ 
lice bias in complaint investigations. Civilian investigators 
may harbor either pro or anti-police bias, depending on 
their own personal background and experiences. Another 
argument is that civilian investigators may also be less 
willing to challenge an officer’s account of events then 
a peer investigator or a police supervisor. Similarly, while 
some former police officer investigators may have the 
ability to investigate without bias, others may approach 
their jobs with the intent or desire to justify or defend 
the conduct of other officers, regardless of whether those 
officers are investigating an agency where they were 
previously employed. 

Independent investigation-focused agencies in large cities 
have also been plagued with budgetary and personnel limitations that have resulted in untimely investigations. The New York 
City CCRB has often been criticized for lack of timely investigations as well as efforts taken by that agency to reduce its workload 
through re-allocation of resources (Clarke 2009). 

Some researchers have argued that while the community may have great conhdence in full investigative models initially, 
community conhdence can wane over time if these models are perceived as not leading to the reforms promised during 
implementation (McDevitt et al. 2005: 5). For example, the public may expect that more citizen complaints will be sustained 
and stronger punishments imposed after full investigative oversight models are implemented. However, there is currently 
no systematic evidence to support this expectation and it is currently unclear what impact full investigative models have 
on patterns in hndings and discipline for police officers alleged to have engaged in misconduct. As a result, disillusionment 
among the public may develop over time when community expectations for change are not met. This is, in part, one of 
the reasons for the recent recommended dissolution of Chicago’s Independent Police Review Authority (Police Account¬ 
ability Task Force 2016). 


Summary of Investigation-focused Agencies 

Key Characteristics 

1. Routinely conducts independent investigations of com¬ 
plaints against police officers. 

2. May replace or duplicate the police internal affairs 
process. 

3. Staffed by non-police, “civilian” investigators. 

Potential Key Strengths 

1. May reduce bias in investigations into citizen complaints. 

2. Full-time civilian investigators may have highly specialized 
training. 

3. Civilian led investigations may increase community trust 
in the investigations process. 

Potential Key Weaknesses 

1. Most expensive and organizationally complex form of 
civilian oversight. 

2. Civilian investigators may face strong resistance from 
police personnel. 

3. Disillusionment among the public may develop overtime 
when community expectations for change are not met. 
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One final criticism sometimes offered in relation to investigation-focused agencies is that they have the potential to undermine 
the responsibility of police chiefs and sheriffs to maintain discipline (McDonald 1981; Prenzler and Ronken 2001.) In police 
agencies where internal affairs units are reduced or eliminated, the opportunity for officers to obtain experience in conduct¬ 
ing personnel investigations and recognizing the extent to which misconduct can negatively impact the agency becomes 
limited or nonexistent. A robust internal affairs process can result in investigators and supervisors who are more committed 


to ensuring a police culture of integrity and 
vestigation-focused models in the U.S. 

accountability within their own 

organization. Table 2 presents examples of in- 

Table 2: Examples of Investigation-focused Models in the U.S. 

Agency 

Jurisdiction 

Website 

Office of Citizen Complaints 

San Francisco, CA 

www.sfgov.org-/occ 

Office of Police Complaints 

Washington, D.C. 

www.policecomplaints. dc. ffov 

Citizen Complaint Review Board 

New York, NY 

www.nvc.ffov/html/ccrb 

Citizens’ Law Enforcement Review Board 

San Diego County, CA 

www.sandieffocountv.gov/clerb.html 

Citizen Police Review Board 

Pittsburgh, PA 

WWW. cprbpgh. org- 

Office of Municipal Investigations 

Pittsburgh, PA 

www.pittsburghpa.gov/omi 


Review-focused Models 


A second category of oversight focuses its work on reviewing the quality of internal investigations, primarily those conduct¬ 
ed by police internal affairs units. Many review agencies take the form of volunteer review boards or commissions and are 
designed around the goal of providing community input into the internal investigations process (PARC 2005). Instead of 
conducting independent investigations, review agencies may evaluate completed internal affairs investigations, hear appeals, 
hold public forums, make recommendations for further investigation 
or conduct community outreach (Attard and Olson 2013). As with 
full investigative agencies, review-focused agencies vary in their 
organizational structure and can perform a range of functions 
(Walker andKreisel 1996; Prenzler and Ronken 2001; Walker 2001; 

Finn 2001; Bobb 2003; Attard and Olson 2013; Ferdik et al. 2013; 

Walker and Archbold 2014). The available literature on oversight 
indicates that review agencies can: 

1. Conduct complaint intake. Some review boards may serve as an alternate intake point for community members who would 
like to file a complaint against an officer, but are uncomfortable visiting a police facility (King 2015). After receiving the 
complaint, review agencies will generally send the complaints to the police or sheriff internal affairs unit for investigation. 

2. Review completed police investigations of externally generated complaints. Review-focused agencies typically only evaluate com¬ 
pleted police internal affairs investigations. They tend to have neither independent investigatory authority nor do they 
typically have the power to subpoena witnesses or records (Walker 2001; Finn 2001; Attard and Olson 2013). Moreover, 
they tend to have authority to only review complaints filed by community members and are less likely to review inter¬ 
nal police complaints (e.g, complaints filed by a supervisor) or critical incident investigations (e.g, officer-involved 
shootings, in-custody deaths) in the absence of a complaint (Bobb 2003). 


“Boards and commissions can greatly enhance community 
confidence in a law enforcement agency. When members of the 
public are empowered to hear concerns and weigh in on 
complaints...public trust is increased. ” (Attard and Olson 
2013: 5) 
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3. Make recommendations to the police executive. After evaluating completed investigations, review agencies may have the ability 
to make recommendations regarding hndings on cases or request that further work be done on cases to remedy deh- 
ciencies in initial internal affairs investigations (PARC 2005: 11). 

4. Hear appeals. Review-focused agencies may also hear appeals from members of the public who were dissatisfied with 
the outcome of their complaint that was filed with, and/or investigated by, the police agency (Finn 2001; PARC 2005). 
In most cases, findings made by a review-focused agency come in the form of recommendations to the police executive. 
In some cases, the agency’s findings may have the force of law. 

5. Gather and review public input. Another common function of review-focused agencies is to hold public forums and gather 
input on the quality of complaint investigations and other law enforcement-related issues. After gathering and review¬ 
ing public input, the review agencies may communicate those concerns to police executives and other policy-makers. 

Potential Model Strengths 

Some researchers have argued that the public may 
perceive review boards and commissions as more 
representative of the community than programs 
that are staffed by full-time professionals (Finn 
2001; Attard and Olson 2013). As such, commu¬ 
nity members may be more likely to perceive the 
oversight entity as supporting and protecting com¬ 
munity interests (Walker 2001). 

Beyond public perception, review-focused agencies 
have the benefit of allowing community represen¬ 
tatives to bring an outsider’s perspective to the com¬ 
plaint investigations process. This may help juris¬ 
dictions identify and correct deficiencies within 
individual complaint investigations (PARC 2005). 

Where review boards have diverse community rep¬ 
resentation, there may be a stronger motivation on 
the part of police investigators to ensure that not 
only is there no bias in the conduct of their investi¬ 
gations, but that any appearance of bias is also 
removed. With respect to the review of policy and 
officer conduct, review boards have the ability to 
identify deficiencies in policy or training as they 
apply to individual cases being reviewed. A diverse 
board will have the ability to provide different per¬ 
spectives on police policy and training and make 
recommendations for change that could result in 
improved community-police relations. 


Summary of Review-focused Agencies 

Key Characteristics 

1. Often focuses on reviewing the quality of completed police 
internal affairs investigations. 

2. May make recommendations to police executives regarding 
findings or request that further investigation be conducted. 

3. Commonly headed by a review board composed of citizen 
volunteers. 

4. Often hold public meetings to collect community input and 
facilitate police-community communication. 

Potential Key Strengths 

1. Ensures that the community has the ability to provide input 
into the complaint investigation process. 

2. Generally the least expensive form of oversight since it 
typically relies on the work of volunteers. 

3. Community review of complaint investigations may increase 
public trust in the process. 

Potential Key Weaknesses 

1. May have limited authority and few organizational resources. 

2. Review board volunteers may have significantly less expertise 
in police issues and limited time to perform their work. 

3. May be less independent than other forms of oversight. 


19 In those cases where an agency can compel a police executive to change a finding, an officer would, as a matter of law, have the ability to appeal a sustained finding of misconduct through 

a civil service process or through the courts. 


28 I 


Civilian Oversight of Law Enforcement | Assessing the Evidence 






Finally, review focused agencies tend to be the least expensive form of oversight. They are often operated by volunteers and 
may have no stand-alone budget (PARC 2005). As a result, this type of oversight is popular in smaller jurisdictions that have 
limited resources. 

Potential Model Limitations 

Like all models of oversight, review-focused agencies may have certain limitations. First, review-focused oversight agencies 
tend to have limited authority and, like full investigative agencies, typically focus on individual case investigations. As a 
result of that reactive focus, their ability to promote large-scale systemic organizational change may be limited (Walker 
2001; PARC 2005). Moreover, review agencies may not have the authority to systemically evaluate police policies or pro¬ 
cedures, make policy recommendations or examine aggregate patterns in officer conduct (PARC 2005: 11). 

Depending on the structure of the review agency, they may be less independent from the police than other oversight models. 
These types of oversight agencies may be more likely to report to the police chief, have a small or no stand-alone budget, 
have limited or no staff support and board members tend to be political appointees (Walker 2001; PARC 2005; Olson 2016). 
Moreover, they may have to rely on the police or sheriffs department for meeting space, administrative support and training. 
Since they do not always have the power to conduct independent investigations, review agencies are also more likely to rely 
on the police or sheriffs department for their information (McDevitt et al. 2005; Olson 2016). Table 3 presents examples of 
review-focused models in the U.S. 


Table 3. Examples of Review-focused Models in the U.S. 

Agency 

Jurisdiction 

Website 

Citizens’ Police Review Board 

Albany, NY 

www.albanvlaw.edu/cprb 

Citizens’ Police Complaint Board 

Indianapolis, IN 

www.indv.gov/egov/citv/dps/cpco 

Civilian Police Review Board 

Urbana, IL 

www.urbanaillinois.us/boards/civilian-police-review-board 

Citizen Review Committee 

St. Petersburg, FL 

www.stpete.org/boards and committees/civilian police 

review committee 


Citizens’ Review Board on Police 
Practices 


San Diego, CA 


wvyw.sandiego.gov/citizensreviewboard 


Since review board members are generally volunteers drawn from a range of professional backgrounds, they may have less 
expertise than paid professional oversight staff and have limited time to perform oversight functions. This may reduce the 
efficiency of a jurisdiction’s oversight function and lead to a shallow impact on the quality of internal investigation (Finn 
2001; Olson 2016). 

Auditor/monitor-focused Agencies 

One of the newest forms of police oversight is the auditor model of oversight referred to by several different names, includ¬ 
ing police monitor^'* or inspector general. As noted earlier in this report, this model of civilian oversight began to develop in 
the 1990s and generally emerged as a type of political compromise to satisfy police-community concerns about bias and 
professionalism (Walker 2006; Walker and Archbold 2014: 180). While local community and civil rights activists tended to 


20 It is important to distinguish between court appointed monitors, who are limited term appointees charged with overseeing the implementation of a court-sanctioned reform agreement, and 

municipal or county civilian monitors who are local oversight professionals or consultants employed by the local jurisdiction. For purposes of this report, the term monitor is used to refer to locally 
employed police monitors. The role of court appointed monitors in promoting police reform is beyond the scope of this publication, but has been explored elsewhere (Davis et al. 2002; Chanin 2015). 
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argue in favor of citizen review boards or full investigative models, police unions tended to be strongly opposed to those 


models. As a result, the auditor model emerged partly as a mechanism for bridging the disparate goals held by the different 


stakeholders to the complaint process (Walker and Archbold 2014). 



citizen oversight agency can 
perform because it is directed 
toward organizational change 
that hoperfully will prevent 
rfuture misconduct” (Walker and 


to have a unique set of goals that distinguish it from investigative and review models of 
oversight (Walker 2001; Finn 2001; PARC 2005; Attard and Olson 2013; Ferdik et al. 
2013; Walker and Archbold 2014). 


1. Audit or monitor internal complaint investigations process. One of the core functions of this 


type of oversight agency is to ensure that a jurisdiction’s processes for investigating Archbold 2014:182). 
allegations of misconduct are thorough, complete and fair (Attard and Olson 2013; 

4). Unlike review or investigative-focused agencies that tend to focus their organizational resources on reviewing or in¬ 
vestigating individual complaints, auditor/monitors are more likely to review broad patterns in citizen complaints or 
internally-generated complaints (e.g, complaints filed by supervisors or other officers) (PARC 2005). While focused on 
systemic change, some police auditor or monitor agencies spend significant resources on the monitoring and auditing 
of individual complaints to ensure integrity in individual investigations, findings and the appropriate imposition of 
discipline. Some police auditors or monitors may also play a broader role in relation to investigations by responding to 
the scene of critical incidents to ensure integrity in those investigations. 

4. Conduct evaluations orf police policies, practices and training. Auditor/monitor-focused agencies may use social science research 
methods or generally accepted auditing methodologies to evaluate the effectiveness of policies, practices or training and 
may issue recommendations for organizational improvement (Walker and Archbold 2014; Olson and Attard 2016). 
Those policy evaluations can be triggered by individual complaints, patterns of complaints or allegations or broader 
investigations into issues that arise outside of the complaint process. In some jurisdictions, auditor/monitor-focused 
models have the authority to conduct data-driven evaluations of police policies, procedures and practices that go beyond 
the internal investigations process (Walker and Archbold 2014: 181; Olson and Attard 2014). 

5. May actively participate in open investigations. Some agencies that fall into the auditor/monitor category may also play a role 
within the complaint investigations process, including the ability to monitor or participate in open internal affairs inves¬ 
tigations (sometimes both internally initiated and civilian initiated), observe investigative interviews and work alongside 
police internal affairs staff as investigations are conducted (PARC 2005; King 2015). Some auditor/monitor models may 
also serve as intake points for community complaints and have the ability to conduct initial intake investigations. 

6. Robust public reporting. Given their focus on bringing about systemic organizational change, auditor/monitor agencies may 
be more likely than other oversight models to have robust public reporting systems (Walker and Archbold 2014). Like 
full investigative agencies, most auditors and monitors issue public reports on aggregate patterns and trends in complaints, 
allegations, findings and discipline and make recommendations for process and organizational improvement. Others 
report on process performance, including workload, investigation thoroughness and timeliness. Some auditor/monitor 
reports include in-depth, data-driven evaluations of police operations, policies, procedures and training (PARC 2005). 

Potential Model Strengths 

Since these agencies tend to focus on exploring patterns in complaints, auditor/monitor models may have broader access to 

police and sheriffs department records, case files and electronic databases than review-focused agencies (McDevitt et al. 2005; 

Olson and Attard 2016). While review boards tend to have only limited access to individual closed internal affairs files, au- 
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ditors/monitors tend to be granted more expansive access to police 
department records (Walker and Archbold 2014). Moreover, auditors/ 
monitors tend to be (or become) policing experts, have larger budgets 
and may have more extensive training than might be found in volun¬ 
teer-based oversight agencies (McDevitt et al. 2005). 

Additionally, it is possible that the auditor/monitor-focused model 
may be more effective at promoting long-term, systemic change in 
police organizations, in part because they can focus on broader 
trends and patterns in complaints and make public recommendations 
for how the police department can improve (Walker and Archbold 
2014). Unlike investigative agencies, auditor/monitor models do 
not generally take the investigations process away from the police 
department, but instead use systematic auditing, monitoring or 
reviews to ensure the department is holding its officers accountable 
(PARC 2005). Auditor/monitor agencies also have the ability to 
track whether police departments implement their recommendations 
and whether those changes have resulted in organizational improve¬ 
ment over time (PARC 2005; Walker and Archbold 2014). 

Some scholars have also argued that the independence of auditor/ 
monitor agencies may also increase their credibility with the public, 
leading to more effective public outreach (Walker and Archbold 
2014; 183). Moreover, the more robust public reporting authority 
and greater staffing resources may enhance the ability of auditor/ 
monitor agencies to conduct effective community outreach when 
compared to review-focused agencies that rely on community vol¬ 
unteers or even independent investigation agencies that focus on 
specihc, individual complaints of misconduct. 

Potential Model Limitations 


Summary of Auditor/ 

monitor-focused Agencies 

Key Characteristics 

1. Often focuses on examining broad pat¬ 
terns in complaint investigations, including 
patterns in the quality of investigations, 
hndings and discipline. 

2. Some auditors/monitors may actively 
participate in or monitor open internal 
investigations. 

3. Often seek to promote broad organiza¬ 
tional change by conducting systematic 
reviews of police policies, practices or 
training and making recommendations for 
improvement. 

Potential Key Strengths 

1. Often have more robust public reporting 
practices than other types of oversight. 

2. Often less expensive than full investigative 
agencies, but more expensive than re- 
view-focused agencies. 

3. May be more effective at promoting 
long-term, systemic change in police 
departments. 

Potential Key Weaknesses 


1. Auditor/monitor focus on examining 
broad patterns rather that individual cases 
be treated with skepticism by some local 
rights activist. 


Local civil rights or community activists may oppose this type of civil¬ 
ian oversight, viewing this model’s reliance on full-time, paid staff with 
skepticism. The focus of auditors/monitors on aggregate patterns in 
complaints and other metrics within law enforcement agencies may 
also leave some community members and civil rights activists dissatisfied, 
since they may desire that discipline be imposed in specihc cases of 
officer misconduct (Walker and Archbold 2014). In fact, the very nature 
of the auditor/monitor model concept may put the police auditor/ 
monitor at odds with community demands or expectations in high 
profile and controversial cases. The concept behind the auditor/monitor 
model is that the office is fair, unbiased and evidence-based in its deci¬ 
sion-making (Walker and Archbold 2001). Such decision-making may result in criticism of the oversight program by the com¬ 
munity, the police or both. Further, to achieve long-term success, a police auditor or monitor may reach compromises on indi¬ 
vidual cases with the police executive to ensure a healthy long-term relationship is developed between the agencies. In some 


2 . 


3. 


Conducting broad, systematic policy 
evaluations requires signihcant expertise. 

Most auditors/monitors can only make 
recommendations and cannot compel law 
enforcement agencies to make changes. 
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cases, a monitor or auditor may decide that officers should not be held accountable in a particular case in lieu of systemic policy 
or training curriculum changes that would result in less resistance from the agency being overseen. This can result in an indi¬ 
vidual case not receiving the result demanded by members of the community; even though in the long-term, police practices 
may be improved overall. 

In some cases, an auditor/monitor agency may choose to allow the police executive to take credit for a reform initiative, 
again to maintain long-term relationships with police leadership. Such actions, while they may support positive reform in 
a police organization, may result in a lack of understanding amongst the community as to the actual effectiveness of the 
oversight program. 

Like other models of oversight, most auditors/monitors can only make recommendations and cannot compel law enforcement 
agencies to make changes (Walker and Archbold 2014; 195). One of the core functions of this model is to use a combination of 
rigorous, data-driven analytical methods, evidence-based recommendations and public reporting to promote organizational 
change within police departments. In cases where a police department does not implement oversight agency recommendations, 
the auditor/monitor will traditionally use its public reporting function to inform policy makers and the wider public about the 
department’s decision. While the threat of public exposure may affect police executive decision-making in favor of implemen¬ 
tation in some cases, a department may publicly take the oversight agency to task on some recommendations or risk short-term 
public controversy to avoid implementing recommendations with which they disagree. However, in this model the police exec¬ 
utive remains the decision-making authority on police policy, procedure and discipline, though the auditor/monitor may have 
significant influence as a result of the ability to use “the bully pulpit” in favor of oversight program initiatives. Table 4 presents 
examples of audit/monitor-focused models in the U.S. 


Table 4. Examples of Auditor/monitor-focused Agencies 

Agency 

Jurisdiction 

Website 

Independent Police Auditor 

San Jose, CA 

www.sanjoseca.gov/ipa 

Office of the Independent Monitor 

Denver, CO 

www.denvereov.ore/ oim 


Independent Police Monitor 

New Orleans, LA 

www.nolaipm.gov 

Los Angeles Board of Police Commissioners Office of the Inspec¬ 
tor General 

Los Angeles, CA 

WWW. oig.lacitv. ore 

Office of the Inspector General for the New York City Police 
Department 

New York, NY 

www.nvc.eov/oienvpd 


One final limitation is that the auditor/monitor model is strongly dependent on the quality of the staff hired to do the work 
(Walker and Archbold 2014). Analyzing patterns in complaints, findings, discipline or conducting performance evaluations 
of other police policies and practices requires a high level of technical sophistication and training, as well as a commitment 
to objective, evidence-based evaluation. The hiring of staff without relevant experience or a commitment to objective, dis¬ 
passionate evaluation methods may cause significant tension between the oversight agency and police executives, as well as 
with rank-and-file officers. 

Organizational Structure and EiFectiveness in Citizen 
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Oversight 

In this section, the research team uses data collected from 97 civilian oversight agencies in the U.S. to compare and identify 
patterns in the structure, authority, practices and resources across civilian oversight agencies in the U.S. The goal of this 
comparison is threefold: 

1. Identify National Patterns in Oversight. Identify broad patterns in organizational characteristics across different models of 
oversight. By exploring broad patterns, this report seeks to help local jurisdictions think about how their potential oversight 
agency may ht within the larger held of oversight. 

2. Compare Organizational Characteristics Across Common Oversight Models. Allow for the comparison of organizational character¬ 
istics across different models of oversight. Much of the case-study research has demonstrated that many contemporary 
oversight agencies exhibit hybrid characteristics and do not ht easily within pre-existing classihcation schemes. 

3. Examine Effectiveness in Oversight. Examine the distribution of organizational characteristics at the national level that have 
been associated with effective civilian oversight of police. 

This section of the report has two subsections. The hrst subsection draws from the organizational data collected from the 
97 civilian oversight agencies to examine broad organizational features of oversight in the U.S., including organizational 
goals, organizational age and the types of law enforcement agencies subject to oversight. The second subsection examines 
key indicators of effectiveness across different oversight models, including independence, legal authority, hnancial resourc¬ 
es, political support and public reporting.^' 

Organizational Characteristics 

In this subsection, the research team draws from the survey data to examine patterns in key organizational features of oversight. 

Common Goals of Civilian Oversight of the Police 

Tkn important issue for any jurisdiction that is considering 
implementing or revising an oversight agency is to determine 
the goals for the program. A number of organizational goals 
are commonly cited across a range of academic and policy 
writing on oversight (Walker 2001:55; Finn 2001:6-11; Bobb 
2003; Harris, 2012; Attard and Olson 2013; King 2015; Pren- 
zler 2016; TUpert et al. 2016). Common goals include: 

• Ensure that the police complaints process is accessible to 
all and to remove impediments to the filing of lawful 
complaints. 

• Ensure that internal investigations are fair and thorough, hndings are reasonable and evidence-based and discipline is 
appropriate. 

• Improve public conhdence in the police and local government by demonstrating that internal investigations are fair and 
thorough and hndings and discipline are reasonable and appropriate. 


Most commonly shared goals across all models 
of oversight: 

1. Improve public trust 

2. Ensure accessible complaint processes 

3. Promote thorough investigations 

4. Increase transparency 

5. Deter police misconduct 


21 Care mu.st be taken when applying these types of observational survey data. These data can be used to identif) patterns of association between models of oversight and the characteristics 

of the jurisdiction in which they operate. However, these data do not allow us to establish whether different models of oversight cause the development of different jurisdictional characteristics or whether 
those characteristics are the result of other unmeasured factors. 
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• Enhance the transparency of police organizations by publicly reporting on the department’s efforts in holding its officers 
accountable. 

• Strengthen police organizations by analyzing patterns in complaints and other police-related data to improve policies, 
practices, training and management. 

• Deter officers from engaging in misconduct through the creation of more effective and consistent investigation and dis¬ 
ciplinary processes. 

• Reducing legal liability from officer misconduct. 

• Improve the public’s understanding of police policy, training and practices. 

While these are commonly cited as goals for the implementation of oversight, it was not clear how commonly oversight agencies 
actually subscribed to these goals. As a result, one of the initial sets of questions asked the oversight executives to rate the importance 
of a series of goals for their oversight agencies.^^ 

The top hve most common critical or important goals identihed in the data collected largely mimic the most commonly cited 
goals in the academic literature (See Appendix B. Table El ). Almost all oversight agencies reported that improving public 
trust (98 percent), ensuring accessible complaint processes (93 percent), ensuring thorough investigations (88 percent), increas¬ 
ing transparency (86 percent) and deterring police misconduct (85 percent) were among their most critical programmatic 
goals. However, clear differences between oversight models emerged in relation to other types of organizational goals: 

• Auditor/monitor agencies tended to rate goals associated with policy review, auditing, statistical pattern analysis and issuing 
recommendations as critical/important for their agency.^^ 

• Investigation-focused agencies were more likely than other types of agencies to indicate that improving accessibility of 
the complaint process, ensuring thorough investigations, complainant satisfaction and ensuring complainants received 
justice were critical or important for their agency. 

• Review-focused agencies tended to rate the importance of goals associated with the complaint investigation process highly. 

What Led to the Implementation of Oversight? 

Much of the writing on oversight indicates that it tends to emerge out of local crises involving 
local police and particularly community conflicts over racially disparate policing and excessive 
uses of force (Walker 2001). To examine patterns in the development of oversight, the agen¬ 
cies identified the types of incidents that catalyzed the creation of their oversight agency (see 
Appendix B. Table B2 ). The most common trigger involved force-related incidents (49 percent). 

Just under 30 percent of the agencies indicated that racially biased policing played a role in 
the development of oversight. Notably, only 11 percent of the agencies indicated that they 
implemented their oversight agency as a proactive or preventive policy effort and not the result 
of a specific critical incident. Overall, there were few differences between oversight agencies 
in terms of the incidents that triggered their implementation—they all tended to have similar origins. One important excep¬ 
tion was that review-focused agencies were more likely to report that they were implemented proactively than auditor/monitor 
or investigative agencies. 


Much of the writing on over¬ 
sight indicates that it tends to 
emerge out of local crises in¬ 
volving local police, and par¬ 
ticularly community conflicts 
over racially disparate policing 
and excessive uses of force. 


22 The possible responses ranged from. Critical, Important, Moderate, Minor and Mo Juri.sdiction. 

23 Investigative agencies fell between auditor/monitor and review agencies on this issue, but were closer to auditor/monitor agencies in their identification of these issues as critical/important 
for their organization. 
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Age of Oversight Agencies 


An important and commonly cited concern within oversight has been a lack of stability of the held. As was evident in the 
historical section of this report, many early 20th century forms of oversight failed in the face of resistance by police unions 
and politicians (Walker 2006). However, the organizational data from the civilian oversight agencies suggest that pattern no 
longer holds. Executives reported on how long their agency has been in operation. As shown in Figure 1, the largest propor¬ 
tion of respondents (55 percent) indicated that their agency has been in continuous operation for 16 or more years. This 
Ending is not surprising since the largest increase in the number of oversight agencies occurred in the 1990s. Although a 
large proportion of current agencies were older than 16 years, the held continues to grow, but at a somewhat slower pace 
today than in the 1990s. Approximately 16 percent of respondents indicated that their agency had been in operation for hve 
years or less.^'^ 

Types of Law Enforcement Agencies Overseen 

A large proportion of the oversight agencies indicated that they were responsible for overseeing municipal police departments 
(82 percent). The second largest area of jurisdiction was for county sheriffs (15 percent). Several agencies indicated that they 
were responsible for overseeing multiple jurisdictions (e.g, municipal police and county sheriffs agencies) ( See Appendix B. 
Table B31 . 

While patterns in jurisdiction were similar across oversight models, auditor/monitor-focused models were somewhat more likely 
to report that they had jurisdiction over county-level agencies than either investigation-focused or review-focused agencies. 

Size of Law Enforcement Agencies Overseen 

The largest proportion of oversight programs oversaw law enforcement agencies (45 percent) that had fewer than 500 
sworn employees. However, there was clear variation across the three models of oversight. Review agencies were much 
more likely to report overseeing smaller law enforcement agencies (500 or fewer officers), while investigative agencies and 
auditor/monitor agencies were more commonly found in jurisdictions associated with larger law enforcement agencies 
f See Appendix B. Table 641 . 

Agency Leadership 

Data was provided that helped identify the char¬ 
acteristics of the administrative head of the 
agency. A majority of the agencies (70 percent) 
had either a full or part-time paid executive, 
though there was noticeable variation between 
types of agencies. Auditor/monitor-focused 
models overwhelmingly had paid executives 
leading the organization (78 percent). Investiga¬ 
tion-focused agencies also tended to have full-time 
paid executives leading the organizations (71 
percent), though almost a quarter were led by multi-member boards. In contrast, a majority of review-focused agencies were 
led by multi-member boards (52 percent) ( See Appendix B. Table B5) . 


Figure 1; How long has your agency been in existence? 


16 or more years 
11 to 15 years 
6 to 10 years 
Less than 5 years 



13% 


0 % 20 % 30 % 40 % 

Percentage of Agencies 


50 % 


100 % 


24 There is an important nuance to this pattern. Approximately 29 percent of the oversight agencies that were 15 years or younger had replaced a previously existing oversight body. Only 4 

percent of agencies that were 16years or older had replaced a previous oversight agency. So while the absolute number of oversight agencies continues to grow, it is clear that a not insignificant 
proportion of the new agencies are replacements for previously existing oversight bodies. 
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Core Elements of Successful Oversight 

What combination of organizational components, authority, practices and resources lead to successful civilian oversight of 
law enforcement? While there is some disagreement within the literature, there is a developing consensus around the core 
components of oversight that are necessary to implement and sustain effective oversight (Perez 1994; Walker and Bumphus 
1992; Prenzler andRonken 2001; Walker 2001; Walker 2003; Bobb et al. 2008; Attard and Olson 2013; Walker and Archbold 
2014; King 2015; Alpert et al. 2016). 

Walker and his colleagues (Walker 2003) developed one of the more comprehensive frameworks outlining the key organization¬ 
al components of an effective oversight agency. They developed the initial framework out of a series of conferences with police 
auditors, where roundtable discussions were held regarding the key elements of effective oversight (Walker and Archbold 2014: 
199). As a result of those conferences. Walker published a set of oversight “principles” in a report titled: “Core Principles for 
and Effective Police Auditor’s Office.” (Walker 2003). Since its publication, several authors have sought to extend slightly revised 
versions of Walker et al.’s original 12 principles to other models of oversight (Olson and Attard 2013; King 2015).^^ 

Based on that literature and Walker’s (2003) core principles, effectiveness in oversight can be grouped into 12 related areas: 

1. Independence 

2. Adequate j urlsdictional authority 

3. Unfettered access to records 

4. Access to law enforcement executives and internal affairs staff 

5. Full cooperation 

6. Support of process stakeholders 

7. Adequate resources 

8. Public reporting or transparency 

9. Pattern analysis 

10. Community outreach 

11. Community involvement 

12. Respect for conhdentiality requirements 

For the remainder of this section, the report draws from the organizational data to estimate the frequency with which these 
components are present within the larger oversight field. 

Independence 

A consensus exists in the literature relating to the crucial nature of independence for the implementation of successful 
oversight of law enforcement (Prenzler and Ronken 2001; Walker 2001; Walker 2003; Olson and Attard 2013; (Savage 
2016). The extent to which oversight is independent of police, political actors and other special interests has been argued 
to be strongly related to effectiveness of oversight. Within the literature, several types of independence have been identi- 
hed, though no social science research has examined the impact that different forms of independence have on the effec¬ 
tiveness of oversight. 


25 In addition, a similar guidance documentfor police monitors was developed in 2008 by the PARC (Bobb et al. 2008). While comprehensive, the guidance in this document covers both 

municipal and county monitors and litigation-based monitors. As a result, the guidelines in this document are highly specific and focused on the monitoring role, and less easily adapted to other models of 
oversight. Nonetheless, this set of guidance would be usefulfor jurisdictions considering implementing a civilian oversight agency with a monitoring component. 
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Structural Independence. Structural independence has been defined as one potential element of independence (Perez, 
1994; Walker 2001; Savage 2016). Walker (2001) argues that oversight agencies that are structurally separate from the police 
departments they oversee will be able to operate with more independence than agencies whose work is intertwined or depen¬ 
dent on local law enforcement. There are multiple ways to achieve this: 

• Prenzler (2000; 2016) contends that investigation-focused models that replace police internal affairs units (which he terms 
“civilian control” models) are the most independent forms of oversight, since non-police citizens take control of all the 
different stages of the complaint investigation process. 

• Oversight agencies that report to political bodies separate from those that oversee police agencies may have greater in¬ 
dependence (Prenzler and Ronken 2001; Savage 2016). 

Legal Protection. A second measure of independence relates to the protections offered by the oversight agency’s founding 
legal authority (Walker 2001; Harris 2012). The more difficult it is to eliminate, censor or shutter the oversight agency, the 
more likely it is the agency will be able to act with some measure of independence (Harris 2012). For example, agencies whose 
authority was created through an amendment to a city’s charter or enshrined in ordinance will likely be more independent 
than agencies that were created through an executive policy change (Walker 2001; Ferdik et al. 2013). Table B6 ( Appendix 
B) identihes the different sources of legal authority enjoyed by the oversight agencies that provided data for this report. More 
than two-thirds of the agencies reported that their legal authority was enshrined in either ordinance or charter documents. 

Job Protections. Specihc language can be entered into an enabling ordinance that prevents agency heads from being 
removed except for cause (Walker 2003). Table E7 ( Appendix B) reports the percentage of agencies that had job protections 
for agency leaders. Overall, 78 percent of full and part-time oversight agency leaders were “at wUl.” 

Anti-Influence Language. Agencies can have legal language in their enabling ordinances that prohibits retaliation based 
on the lawful acts taken by oversight agencies in the course of their work (Walker 2003; Attard and Olson 2013). 

Limit Involvement of Police Officers. One controversial approach to independence relates to whether current or former 
police officers should be able to work or volunteer for civilian oversight agencies. Some oversight agencies do not allow current 
or former police officers to work or volunteer for their agency. Other agencies may employ current or former officers who 
have worked for other jurisdictions, but not for the police agency being overseen. Other departments allow or require the 
oversight agency to use retired or currently serving officers. There is no social science research that systematically establishes 
whether allowing police officers to work or volunteer for oversight increases or decreases the quality of an oversight agency’s 
work (Savage 2016) or affects the public’s perception of its independence. 

The research team asked 97 civilian oversight agencies whether they allowed former police officers to serve as their agency 
executive or director. TUtogether, 50 percent of the agencies responded that a former police officer could serve as their agency 
executive/director. An additional 36 percent reported that a former police officer could serve as the executive/director “under 
some circumstances” ( Appendix B. Table B8) . 

Adequate Jurisdiction and Authority 

An important component of effective oversight identihed in the research and 
policy literature related to the jurisdictional authority granted to oversight agen¬ 
cies (Attard and Olson 2013). Agencies need to have adequate jurisdiction that 
will allow them to achieve their organizational goals. Of course, the goals for 
oversight can vary signihcantly between jurisdictions depending on local issues, 
the political climate and a jurisdiction’s hnancial resources. Little is known about whether different types of jurisdiction or 
authority have a greater or lesser impact on patterns in officer misconduct or organizational reform. 


Little is currently known about whether 
different types of jurisdiction or authority 
have a greater or lesser impact on patterns 
in officer misconduct or organizational 
reform. 
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Adequate Jurisdiction. One issue local jurisdictions need to decide on is whether the local oversight agency will have the 
ability to review allegations of misconduct that emerge from sources outside of citizen complaints. Table B9 ( Appendix E ) 
reports the core areas of jurisdictional authority across the three different models of oversight. Almost all of the agencies 
submitting data indicated that they have jurisdiction in relation to citizen complaints. A majority of the agencies stated they 
“always/sometimes” have jurisdiction in relation to: 

• Officer-involved shootings. 

• Serious force. 

• In-custody deaths. 

There were some differences, however, between oversight models in other areas. For example, auditor/monitoi^focused 
agencies were more likely than other models to report that they had jurisdiction in relation to officer-initiated complaints, 
supervisor complaints, high-speed pursuits and criminal investigations, as compared to review and investigative agencies. 

Adequate Authority. A second critical issue in this area relates to authority that oversight agencies have in relation to the 
handling of complaints. Given that most oversight agencies perform an oversight function in relation to citizen complaints, the 
agency executives provided data to identify the specihc role their agency plays in relation to the processing of citizen complaints. 

In looking at Table E10 ( Appendix E) . it is clear that there are important differences among the oversight models in terms 
of the role they play in the handling of citizen complaints. 

One remarkable pattern related to the breadth of the organizational authority enjoyed by auditors/monitors. Almost all of 
the auditor/monitor agencies reported that they were able to; 

• Review police complaint investigations (96 percent). 

• Audit police complaint investigations for compliance with investigative standards (92 percent). 

• Monitor open police complaint investigations (92 percent). 

• Refer complaints back to internal affairs for additional investigation (78 percent). 

• Conduct independent investigations (61 percent). 

• Recommend findings on investigations (70 percent). 

• Recommend discipline (61 percent). 

Investigative agencies reported somewhat more focused forms of authority than auditor/monitor agencies and tended to 
report that they were able to; 

• Conduct independent investigations (100 percent). 

• Decide how complaints would be handled (97 percent). 

• Classify complaints (94 percent). 

• Conduct intake investigations (94 percent). 

• Review completed police complaint investigations (67 percent). 

• Recommend and issue findings (84 percent and 88 percent). 

Review agencies reported the most focused forms of authority and reported strong involvement in: 

• The review of police complaint investigations (81 percent). 

• Referring complaints back to internal affairs for further investigation (65 percent). 
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Unfettered Access to Records 

An organizational power commonly identified as a critical component of effective oversight relates to an oversight agency’s 
access to police department records (PARC 2005; Olson and Attard 2013; King 2015). Both Attard and Olson (2013) and King 
(2015) identify access to records as one of the most important components of effective oversight. Similarly, Walker (2003) includes 
unfettered access as one of the core principles for an effective police auditor (Walker and Archbold 2014: 200). 

To explore the extent to which contemporary oversight agencies have access to police records, the data received from the 97 
civilian oversight agencies included a series of questions that asked executives about their level of access to police records. 
Table B11 l Appendix B l identifies the proportion of oversight agencies reporting that they always or sometimes have access 
to specific kinds of records. 

More than two-thirds of the oversight agencies reported that they always or sometimes had access to closed internal in¬ 
vestigations. There was substantial variation; however, across the different models of oversight in relation to records access. 
Auditor/monitor agencies tended to report high levels of records access, with more than three quarters reporting that 
they always or sometimes had access to all internal affairs files and internal affairs databases. More than half of the auditor/ 
monitor agencies also reported having access to all police databases. In contrast, 38 percent of review-focused agencies 
had access to aU internal affairs records, 30 percent reported access to internal affairs databases and 19 percent reported 
access to all police databases. 

Investigative agencies were more likely than other models to report the ability to subpoena witnesses (56 percent) and records 
(59 percent). 

A large proportion (74 percent) of all agencies indicated that they had access to police body-worn/in-car video. 

Access to Law Enforcement Executives and Internal Affairs St(^ 

A fifth component of effective oversight relates to the extent to which oversight actors have 
the ability to influence appropriate law enforcement officials. Walker (2003) notes that over¬ 
sight agencies should have “...direct access to the chief executive of the law enforcement 
agency under review.” (p. 7). From the perspective of the authors, while gaining access to the 
police chief or sheriff is important, it is also critical that the police chief or sheriff and their 
internal affairs staff are willing to both consider and implement the oversight agencies rec¬ 
ommendations. Given that, the organizational data collected included information about 
whether: (1) police executives “listened carefully” to the recommendations made by the 
oversight agency; and (2) police departments were willing to implement the recommendations 
made by the oversight agency. 

Listening to oversight recommendations. There was little variation between models of oversight in relation to 
whether police executives considered the views of oversight actors. Roughly 78 percent of auditor/monitor, investigative 
and review-focused agencies indicated that police executives listened carefully to their recommendations ( see Table B12 
in Appendix B l. 

Implementing oversight recommendations. There was more variation in relation to whether police executives were 
willing to implement the recommendations made by oversight agencies ( see Table B13 in Appendix B ). As shown in Figure 
2, auditor/monitor agencies were much more likely to report that police executives very frequently or frequently implement¬ 
ed their recommendations (72 percent), while 42 percent of the responding investigative agencies and 34 percent of the review 
agencies indicated that police or sheriffs departments frequently implemented their recommendations. Some of this difference 


To be effective, oversight agencies 
must have regular access to police 
executives. However, effectiveness 
hinges not just on access, but also 
on the willingness of police 
executives to both consider and 
implement the recommendations 
made by oversight agencies. 
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may relate to the type of work different agencies tend to perform. Auditor/monitor agencies are more likely to have special¬ 
ized staff who focus on analyzing patterns in complaints and other police data and they are more likely to issue special topics 
reports as a result of that work (which Harris (2012) describes as a “holistic’ approach). It is possible that that type of analyt¬ 
ical work makes it more likely that police departments will implement recommendations made by oversight. More systemat¬ 
ic comparative research is needed before strong inferences can be drawn on this issue. 

Full Cooperation 

Another component of effective oversight 
relates to the ability of an oversight agency 
to gain the cooperation of police agency 
executives and officers (Walker 2003; 

Walker and Archbold 2014). Some agencies 
attempt to achieve voluntary cooperation 
by developing a collaborative working re¬ 
lationship with the departments they 
oversee and attempting to engage depart¬ 
ments, officers and unions within the over¬ 
sight process (MiUer 2002). Other jurisdictions attempt to strengthen cooperation by having legislative requirements for 
cooperation built into the oversight agency’s enabling ordinance, charter or statute. To date, there is no social science 
evidence that systematically tests whether one or both of these approaches improve the effectiveness of oversight. 

There are two common variations in legislated cooperation requirements. First are general cooperation requirements that 
direct all law enforcement employees within a particular jurisdiction to cooperate with the oversight agency during the course 
of its work, but do not clearly articulate sanctions for non-cooperation. A second type of cooperation requirement buUds in 
the possibility of a sanction for non-cooperation, where law enforcement employees are required to cooperate with the over¬ 
sight agency as a condition their employment (i.e., non-cooperation could constitute grounds for termination). 

The organizational data received from the civilian oversight agencies f see Table E14 in Appendix E l showed investigation-fo¬ 
cused agencies tended to have more robust cooperation requirements than other forms of oversight (see Figure 3). Altogeth¬ 
er 73 percent of investigative agencies reported that their jurisdiction included a general cooperation requirement, while 52 
percent of auditor/monitor agencies and 40 percent of the review agencies reported a cooperation requirement. A much 
smaller proportion of the agencies reported that their enabling ordinance/statute made cooperation a condition of employ¬ 
ment. Less than half of investigative agencies indicated that cooperation was a condition of employment for officers or 

1 20 percent of review agencies had the same authority. 

Support from Process Stakeholders 

A sixth key component of effective oversight relates to the 
importance of process stakeholder support for the oversight 
agency’s work. A lack of political support has the potential 
to undermine an otherwise well designed system of over¬ 
sight. Government officials and office holders, if not sup¬ 
portive of an oversight agency, can reduce its effectiveness 
in a variety of ways, including by failing to provide the 
agency with adequate resources or authority or by appoint¬ 
ing ineffective managers or board members (Attard and 


command staff. Only 26 percent of auditor/monitor agencies 

Figure 3; Percent of Oversight Agencies Reporting that their 
Jurisdiction has a General Cooperation Requirement 
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Olson 2013: 6). Similarly, opposition from police unions, local district attorneys or police executives has the potential to 
complicate the work of oversight agencies (Wilson and Buckler 2010). While conflict is not necessarily avoidable in all cases, 
the extent to which an agency can achieve its organizational goals will be partly dependent on whether it has support from 
local ofhce holders, the police executive and other key process stakeholders. 

Table E15 l Appendbc E l describes the percentage of agencies reporting that key process stakeholders have shown very strong 
or strong support for oversight. A large percentage of the responding agencies across all models reported very strong or strong 
support from most key process stakeholders, including police executives, city attorneys or county counsels, mayors, city man¬ 
agers, city council and the local civil rights community. Reported support was noticeably lower among police unions. 

There was little variation in reported support across the three models of oversight,^® with three exceptions. Review-focused 
agencies were somewhat more likely to report support from police unions. Auditor/monitor agencies were less likely to indi¬ 
cate support from city attorneys/county counsels, but were slightly more likely to report support from the local civil rights 
community. Investigative agencies reported the least support from police unions. 

Adequate Resources 

A seventh organizational component critical for the effectiveness of oversight relates 
to the resources available to the oversight agency. In general, an oversight agency’s 
resourcing is considered one of the most important potential indicators of effec¬ 
tiveness (Finn 2001; Walker 2001; Walker 2003; Attard and Olson 2013). In dis¬ 
cussing resources, most scholars and policy experts distinguish between the impor¬ 
tance of having a budget that matches an agency’s goals and the presence of 
professional staff who have the time and expertise to support the work of the oversight agency (Finn 2001; Walker 2003). For 
example, Olson (2016) recently conducted a limited examination of one review-focused agency and found that local agency 
was unable to perform certain important oversight functions or fully engage with important stakeholders to the complaint 
investigations process due to limited budgetary and financial resources (Olson 2016: 6). 

Even though there is some commonsense appeal to the notion that budgetary and staffing resources are related to effective¬ 
ness, no social science research has systematically examined the impact that varying budgets have on the effectiveness of 
oversight. That said, there is noticeable variation between models of oversight in two areas: budgets and staffing. 

Adequate Budget. Table B16 ( Appendix B) examines patterns in oversight agency resources. It shows a breakdown in the 
annual budgets for the agencies across the three model categories. As was seen in the review of the previous literature on 
models of oversight, review-focused agencies tended to have much smaller budgets than auditor/monitor and investigative 
oversight agencies. Just under 60 percent of the review agencies reported having no operating budget and another 20 percent 
of review agencies reported budgets of less than $ 100,000 per year. In comparison, roughly half of the auditor/monitor and 
investigative agencies had budgets that were greater than $500,000 per year. Overall, this pattern in funding can be at least 
partially explained by the fact that investigation-focused and auditor/monitor-focused agencies are more likely to be imple¬ 
mented in jurisdictions with bigger populations and larger law enforcement agencies. In addition, where programs have more 
limited mandates, it can be expected that the resources needed to achieve that mandate would require less funding. 

Adequate Staffing. Table B17 ( Appendix B) evaluates patterns in the staffing resources available to oversight agencies across 
the three models. The table shows the distribution of full-time, paid staff across the three different types of civilian oversight. 
As was seen with agency budgets, review-focused agencies tended to have the fewest staff, with 61 percent of the review 
agencies reporting that they had no full-time, paid staff. Investigative agencies tended to report a larger number of full-time. 


26 It is important to note that there was a noticeable increase in the number of agencies that chose not to respond to questions about stakeholder support as compared to other questions. 


Many oversight agencies reported very 
strong or strong support from stakeholders 
such as mayors, city council and the local 
civil rights community, but less support 
from police unions. 
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paid staff with 21 percent of the investigative agencies reporting they had 16 or more full-time staff members. Auditor/ 
monitors fell in-between review and investigative agencies, with 26 percent of auditor/monitors reporting no full-time staff 
and an additional 43 percent indicating that they had one to hve full-time staff. As with patterns in relation to budgets, vari¬ 
ation in staffing is at least partly the result of jurisdiction size—that is, investigative agencies and auditor/monitor agencies 
are more likely than review-focused agencies to oversee large police departments in bigger cities. 

Public Reporting or Transparency 

The eighth component of effectiveness in oversight relates to public reporting and the level of transparency that the oversight 
agency brings to the complaint handling process and other areas of police operations (Walker 2001; Finn 2001; Attard and 
Olson 2013). As we saw with the data on agency goals, increasing the transparency of the complaint investigations process 
was one of the most commonly cited organizational goals identihed by the civilian oversight agency executives. While there 
is no comparative, systematic social science research on the impact of public reporting on community perceptions relating 
to the transparency of the internal investigations process, the civilian oversight executives provided data regarding their 
agency’s public reporting and data analysis practices. 


Issue Public Reports. ( See Table B18 in Appendix B) . Most civilian oversight agencies reported that they publish public 
reports (78 percent), although there was slight variation among oversight agencies that provided data for this report, with a 
slightly smaller proportion of review boards reporting that they publish reports (69 percent) as compared to auditor/monitor 
(80 percent) and investigative agencies (85 percent). 


Pattern Analysis 

Another potentially important element of an oversight agency’s work is the extent to which 
it analyzes and reports on aggregate patterns in relation to complaint handling processes 
and outcomes. Police departments have traditionally been reluctant to report publicly on 
patterns in internal investigations and an important role of oversight has been to use public 
reporting to inform the community about patterns in complaints, investigative outcomes 
and indicators of process or procedural effectiveness (e.g, timeliness of the process) (Walker 
2001; Finn 2001; Attard and Olson 2013). Given this, the civilian oversight agencies shared 
information that help examine data management and pattern analysis practices across 
different models of oversight (see Table B19 in Appendix B) . 


A larger proportion of auditor/ 
monitor agencies reported 
conducting analyses of patterns 
in officer-involved shootings, 
officer discipline, in-custody 
deaths and police data relating 
to stops, searches and arrests. 


The most common types of issues analyzed very frequently or frequently by oversight agencies were complaint handling 
workload (i.e., number of complaints), allegations and hndings on allegations. Only a small proportion of agencies (20 
percent) frequently conducted analyses of issues of critical concern to the public and communities of color, such as 
patterns in police stops, searches and arrests. 


There were a number of noticeable differences across types of agencies in relation to the use of pattern analysis. A larger 
proportion of auditor/monitor agencies reported conducting analyses of patterns in officer-involved shootings, officer disci¬ 
pline, in-custody deaths and police data relating to stops, searches and arrests. 


Community Outreach 


A tenth component of civilian oversight relates to the ability of agencies to conduct outreach to community members and 
other process stakeholders. Overall, there is a strong consensus in the research and policy literatures that community outreach 
is an essential element of any effective oversight agency (Walker 2001; Finn 2001; Stewart 2006; Walker and Archbold 2014). 
While the resources that oversight agencies have can vary substantially, outreach activities provide oversight agencies with 
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the opportunity to: 

1. Publicize the different processes for handling complaints; 

2. Reach out to disenfranchised members of the community who might be fearful or distrustful of the police; 

3. Talk with that the community about police policies, procedures or training; and 

4. Gather input from a range of community members and groups. 

A successful oversight agency will typically include a robust outreach program. In terms of the research literature, a few 
authors have identihed several components of an effective outreach program, including holding public outreach events 
involving different community groups, engaging in a range of diverse outreach activities and making attempts to conduct 
outreach with difficult-to-reach or underserved populations (Walker 2001; Stewart 2006; King 2015). 

Outreach to Groups. The first element of an outreach program relates to the whether the oversight agency is engaging 
in outreach activity at aU (Walker 2001). Table B20 ( Appendix B) identihes patterns in outreach across oversight models. 
The more common groups targeted for community outreach included community leaders (77 percent); neighborhood 
associations (69 percent); policy-makers (65 percent); and police command staff (59 percent). As with other outreach ac¬ 
tivities, review-focused agencies reported conducting somewhat less outreach to stakeholders than either auditor/monitor 
or investigative agencies. 

Outreach Activities. An additional important component of outreach relates to the type and range of activities (Walker 
2001). To explore the kinds of outreach activities that different models of oversight were using, the civilian oversight agencies 
provided data regarding their outreach activities (see Table 
E21 in Appendix E ). The operation of an agency website was 
the most common form of outreach (69 percent), followed by 
tabling at community fairs (48 percent), offering complaint 
forms at multiple community locations (46 percent) and holding 
community forums (45 percent). 

Outreach to Special Populations. An additional measure 
of outreach effectiveness relates to the extent to which oversight 
agencies direct outreach resources toward special populations 
(see Table E22 in Appendix E ). The civilian oversight agencies 
provided information on whether they translated any of their 
agency documents into languages other than English as one 
measure of this form of outreach. Just under half of aU agen¬ 
cies reported translating complaint forms (44 percent) and 
agency brochures (48 percent) into languages other than 
English. Only a small proportion of agencies translated web¬ 
pages or agency reports. Review agencies were noticeably less 
likely to report that they translated outreach documents than 
other models of oversight. 

Community Involvement 

Tkn eleventh component of effective oversight is community involvement. McDevitt et al. (2005) note that it will likely be impos¬ 
sible for effective oversight to be implemented without sufficient community involvement. Involving community stakeholders in 


Reaching Out to Special Populations 

The types of outreach that oversight agencies conduct 
win depend not just on their resources, but also on the 
specific social and cultural characteristics of populations 
in which they operate. Some oversight agencies have 
developed sophisticated community outreach strategies 
tailored to their local communities. 

For example, the San Jose Office of the Indepen¬ 
dent Police Auditor makes many of their key over¬ 
sight documents available in English, Spanish, and 
Vietnamese. They also maintain an up-to-date Face- 
book® page, distribute a resource guide to local students, 
developed a training video for teachers and produced a 
local cable access television show that explored import¬ 
ant law enforcement and oversight issues facing the San 
Jose community. Example material can be found here: 
WWW, sanjoseca.gov/index. aspx?nid=200 
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the process allows the oversight agency to identify and address the key accountability issues that the jurisdiction is facing (Mc- 
Devitt el al. 2005: 7). Different types of civilian oversight agencies involve community members in a wide variety of ways—some 
oversight agencies involve community volunteers in the core work (review-focused agencies), while other highly professionalized 
agencies may have community members sitting on governing boards, police commissions or advisory boards. However, there is 
no current empirical research that examines whether one approach is more effective than another. 

Given the arguments for the importance of involving community members in oversight, it is not surprising that a very 
large proportion of the oversight agencies who provided data indicated that they operated with a board composed of 
community members (See Appendix B . Table 6231 . 

Respect for Confidentiality Requirements 

Some oversight practitioners have argued that another important component of effective oversight relates to the ability of 
civilian oversight agencies to protect the conhdentiality of employee personnel records (Walker 2003). Many states and local 
jurisdictions have legal rules that prevent the disclosure of certain kinds of personnel records. While these vary considerably 
by state, the ability of a civilian oversight agency to be effective in its work will depend, at least in part, on its ability to respect 
conhdentiality rules (Attard and Olson 2013). Failure to respect state statutes relating to conhdentiality may constitute a 
serious violation of professional ethics, undermine trust between the oversight agency and the local police department and 
may cause the oversight agency to lose access to conhdenrial records. To date, no systematic comparative research has exam¬ 
ined how effectively civilian oversight agencies have been at ensuring the conhdentiality of the records they handle. 
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Trending Issues in Civilian Oversight 

In the previous section, the team drew from the research literature and organizational data provided by 97 civilian oversight 
agencies to explore 12 broad organizational components identihed as being critical for ensuring effectiveness in oversight. In 
this section of the report, the team explores some of the key recent developments in civilian oversight of police. 

Using Performance Metrics to Capture Oversight Effectiveness 

Over the last ten years there has been a growing interest in the question of how the effectiveness of police oversight agen¬ 
cies can be measured (Filstad and Gottschalk 2011; FauU 2013). More specihcally there is a developing body of research 
focused on exploring whether “performance measures” can be used to capture the impact oversight agencies have in their 
work (Brereton 2000; Miller 2002; Prenzler and Lewis 2005; Mohr 2007; Filstad and Gottschalk 2011; Faull 2013). Per¬ 
formance measures are indicators of how successful an organization has been in achieving an organizational objective 
(Hatheld 1994). Performance measures are typically expressed statistically (i.e., as numbers or rates) and usually focus on 
the impact, timeliness, quality or productivity of an organization. Put another way, performance measures should teU the 
public, managers and policymakers how well an organization is doing, whether it is achieving its organizational mission 
and whether changes are necessary to improve the organization’s effectiveness. 

There is currently no consensus on how to measure organizational performance in the held of civilian oversight (Faull 2013). 
Evaluating the impact of oversight can be difhcult for several reasons. First, many oversight agencies make recommendations 
based on their work reviewing, monitoring or auditing police internal invesrigations. Thus, their work is intertwined with 
internal police accountability systems (Brereton 2000; Prenzler and Lewis 2005). Second, evaluating the distinct impact that 
oversight has is complicated by the fact that there are a large number of variables involved and patterns in complaint-related 
data can be interpreted in multiple ways (Walker and Bumphus 1992; Walker 2001; Prenzler and Lewis 2005). For example, 
increases in the number of citizen complaints tiled could indicate that officer misconduct is increasing or it could indicate 
that community members are more willing to file complaints due to increasing confidence in the process. Finally, there is wide 
variation among oversight agencies in their organizational structure, jurisdiction, resources, data collection practices and 
authority, making comparison between and among agencies difficult (Walker 2001; Prenzler and Lewis 2005). 

Even if there is no current agreement about how best to measure organizational effectiveness in police oversight, a number 
of studies have sought to explore the value of different performance measures (Walker 2001; Prenzler and Lewis 2005; Filstad 
and Gottschalk 2011). David Brereton (2000) argues that there are four key questions that should be asked when assessing 
the performance of civilian oversight agencies: 

• How successful have oversight agencies been in improving the process for investigating complaints against the police? 

• Are civilian oversight agencies more successful at increasing public satisfaction than systems without oversight? 

• Has the implementation of civilian oversight agencies improved police practices and behavior? 

• Has the implementation of oversight negatively impacted officer morale or police operations (Brereton 2000: 106; see 
also FauU 2013)? 

Prenzler and Lewis (2005) used an analysis of public reports and a brief survey to explore the performance measures used 
by civilian oversight agencies in Australia in the early 2000s. They offered a number of conclusions. First, they observed that 
most Australian oversight agencies were attempting to evaluate their performance, but that they only used relatively simple 
measures of effectiveness, such as: 

• The number of complaints filed and closed per year. 
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• Time taken to close complaints. 

• Patterns in outcomes, such as substantiated complaints, referrals, mediations and discipline. 

Prenzler and Lewis (2005) contend that while these simple measures are useful for capturing an agency’s workload, they do 
not capture much about an agency’s performance or impact. They then identihed a large number of other measures used 
by oversight agencies to measure performance (p. 78-82): 

• Trends in reported police misconduct (e.g, number of cases hied). 

• Workload measures (e.g, number of open cases, cases reviewed, number of investigations conducted, number of 
appeals, identihcation of case dehciencies). 

• Patterns in hndings and outcomes on case investigations. 

• Timeliness and process efficiency measures (e.g, number of cases assessed in a given time period). 

• Public opinion survey measures (e.g, satisfaction rates for complainants and officers, community conhdence in the 
police). 

• Review and recommendation effectiveness (e.g, number of recommendations made, number implemented). 

• Accessibility measures (e.g, accessibility of the office to the public). 

More recently, other scholars have used slight different variations on the performance indicators identihed by Prenzler and 
Lewis (Filstad and Gottschalk 2011: 102-104; Mohr 2007; FauU 2013). However, there is a general consensus that strong 
caution should be used when attempting to use “sustain” or “conviction” rates to measure an oversight agency’s performance 
or to compare performance across oversight systems.Walker (2001) points out a number of pitfalls associated with using 
sustain rates, including an absence of standards for “acceptable” sustain rates; the lack of understanding of potential differ¬ 
ences between complaints hied with oversight agencies and complaints hied with police department; the fact that investigations 
are only one potential function of oversight and the presence of profound data problems that make calculating; and com¬ 
paring sustain rates unworkable (Walker 2001: 120-122 and 134-135). 

Problem-oriented Approaches to Civilian Oversight 

A second trending issue has to do with an increasing focus on shifting some of the emphasis within oversight away from re¬ 
active forms of case review and investigation and toward more proactive reform efforts (Porter 2013; Harris 2012). Part of 
this trend has been visible since the 1990s with the development of auditor/monitor models of oversight (Bobb 2003). 
However, there has been a developing focus on adapting problem-oriented policing strategies for use in civilian oversight. 

Problem-oriented policing has been an important approach to law enforcement for more than 30 years. In this approach, 
patterns of crime and disorder are analyzed carefully to develop new programs that can reduce the incidence of the identi¬ 
hed crime (Goldstein 2003; Braga 2008; Porter 2016). This approach to law enforcement is notable because it is designed to 
be both preventive and collaborative. That is, it is designed to reduce future crime problems and bring all of the stakeholders 
together to develop a solution (Goldstein 2003). Even though this strategy has been popular with law enforcement, it has not 
been widely adopted for use in civilian oversight of law enforcement. 

Recent research by Porter (2016) indicates that it is possible to adapt problem-oriented approaches to increase the effective¬ 
ness of civilian oversight.^® Often referred to as the SARA problem-solving model, this strategy has four stages: 


27 In this context, a sustain rate would be calculated as the percentage of complaints in a given period that result in one or more sustained findings. 

28 See also Walker (2001) for an early statement on the value of problem-oriented policing for civilian oversight of law enforcement. 
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1. Scanning: A specific problem is identified; 

2. Analyzing: Data is collected and analyzed so the breadth and causes of the problem can be identihed; 

3. Responding: A potential solution for the problem is developed and implemented; and 

4. Assessing: A rigorous foUow-up examination is conducted to ensure the new program is working and to determine 
whether changes to the intervention are necessary (Braga 2008; Porter 2016: 172). 

In arguing for the potential value of the problem-solving model for oversight, Porter (2016) offers two case studies of oversight 
programs where the SARA model was used successfully First, she reviewed a collaborative project on use of force undertak¬ 
en by the Portland Independent Police Review Division (IPR) and the Portland Police Bureau. She reported that this collab¬ 
orative task force conducted a deep analysis of uses of force and made 16 recommendations for changing the agency’s pol¬ 
icies, procedures, practices and training around use of force. The task force then conducted a foUow-up assessment and 
reported that uses of force, excessive force complaints and injuries to officers and community members had declined after 
the recommendations were implemented (Porter 2016: 178; citing to Force Task Force 2007 and 2009). 

In a second case study. Porter reviewed a collaborative problem-solving effort that focused on TASER use in Australia. Porter 
(2016) reported that the project, which was undertaken by the Queensland, Australia Crime and Misconduct Commission 
(CMC) and the Queensland Police Service (QPS), resulted in a measurable decline in the use of TASERs by the police over 
time (Porter 2016: 178). 

Expanding Alternative Dispute Resolution 

The use of mediation by civilian oversight to resolve police complaints has become an increasingly popular strategy over the 
last 15 years. The organizational data collected from 97 civilian oversight agencies for this project included information on 
whether they mediate complaints against the police. Altogether, 29 agencies reported that they had a mediation program, up 
from 20 agencies in 2008 (PARC 2008). 

Unlike traditional investigation-focused forms of complaint processing, mediation is often described as the informal resolution 
of a dispute through a conhdential face-to-face meeting facilitated by a neutral third party (Walker and Archbold 2000). While 
different jurisdictions implement mediation in slightly different ways, one of the themes that ties almost aU programs together 
is the goal of using facilitated, face-to-face meetings between police officers and community members to resolve less serious 
complaints, such those that involve discourtesy or misunderstandings about police procedure (Walker et al. 2002; PARC 2008; 
Schaible et al. 2013). Underlying this theme is the belief that mediation can help overcome some of the limitations of tradition¬ 
al police complaint handling procedures for certain kinds of cases. For example, traditional complaint investigations processes 
for less serious complaints can be inefficient, rarely result in sustained hndings or the imposition of discipline and tend to leave 
both complainants and officers highly dissatisfied (PARC 2008). In contrast, mediation may be a more cost-effective and efficient 
method of resolving minor complaints than more traditional complaint investigation processes (Walker et al. 2002). Moreover, 
the use of mediation is designed to help agencies improve officer and complainant satisfaction by bringing community members 
and police officers together in a non-adversarial setting to discuss their concerns (Quinn 2006). 

There is a growing body of research that provides support for the argument that mediation can have some positive benehts 
for complainants and officers, especially in relation to complaints alleging less serious misconduct (Walker et al. 2002; PARC 
2008: 45). Schaible et al. (2013) used attitudinal survey data collected in one jurisdiction to compare patterns in attitudes 
between complainants and officers who went through a mediation process to a comparable set of complainants and officers 
whose complaints were handled through a traditional investigation process. The study found that both police and commu¬ 
nity participants in the mediation program reported signihcandy higher levels of satisfaction than those who went through 
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the traditional process. Moreover, the study found that mediation had even stronger effect on Latino complainants and female 
police officers. 


For local jurisdictions considering implementing a mediation program, several U.S. oversight agencies operate well-known 
and long-running community-police mediation programs. Each of these agencies publish detailed, useful information about 
the philosophy and operation their individual programs (Table 5 for links to each program’s mediation webpage). 


Table 5. U.S. Civilian Oversight Agencies 

that Operate Model Mediation Programs 

Jurisdiction 

Oversight Agency 

Sworn 

Officers 

Mediations 
Completed 
in 2015 

Mediations 
per 1,000 
Officers 

Agency Website: 

New York, NY 

Civilian Complaint 
Review Board 

34,500 

205 

5.9 

www.nvc.gov/html/ccrb/html/ 

mediation/mediation, shtml 

San Francisco, CA 

Office of Citizen 
Complaints 

2,208 

45 

20.4 

www.sfa'ov.ors'/occ/mediation 

Washington, D.C. 

Office of Police 
Complaints 

3,789 

42 

11.1 

www.policecomplaints.dc.gov/ 

service/mediation-service 


Office of the 

Independent Monitor 




www.denvergov.org/ content/ 

Denver, CO 

1,442 

38 

26.4 

denvergov/en/office-of-the-inde- 

pendent-monitor/mediation.html 

Note: This table and mediation data were adapted from The Denver Office of the Independent Monitor’s (2016), 2015 Annual Report (pp. 45). 
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Conclusions and Recomniendadons 

To help municipal executives, policy-makers, community leaders and the general public consider some of the important issues 
currently facing police oversight, this report conducted a broad review of the current state of knowledge regarding civilian 
oversight of law enforcement in the U.S., drawing from a review of the research literature on oversight and organizational 
data provided by 97 police oversight executives. 

There has been a fair amount of writing on civilian oversight and it is currently a topic of substantial national interest. Un¬ 
fortunately, there has been far less systematic, comparative social science research on the effectiveness of oversight. That said, 
there are a number of resources that can act as a jumping-off point for jurisdictions considering implementing civilian over¬ 
sight or who are evaluating the effectiveness of their current oversight mechanisms. 

The early seminal work by Walker (2001) and Finn (2001) still stand as critical resources for jurisdictions weighing the costs 
and benehts of implementing civilian oversight. More recent research by Prenzler and his colleagues (Prenzler and den Heyer 
2016) update what is known about the state of the held and offer useful information about novel developments outside of 
the U.S. Finally, there have been a series of recent program evaluations of agencies that consider some of the strengths and 
weaknesses of individual civilian oversight agencies in Chicago (Police Accountability Task Force 2016), King County (An¬ 
derson et al. 2015), New Orleans (PARC 2016), San Francisco (2016) and Spokane County (Olson 2016). 

Given that there has been relatively little research on civilian oversight, this project drew strongly from data provided by 
civilian oversight agencies to identify important issues within the held. Using those data, it was possible to catalogue a 
number of key issues that jurisdictions may want to consider when evaluating their oversight mechanisms. 

Oversight Is Common in the U.S. 

Patterns in the development and persistence of oversight indicate that civilian 
oversight has been increasingly institutionalized as a regular feature of policing 
in cities and counties across the U.S. In 1975, there were seven oversight agencies 
in the U.S. That hgure grew to over 100 agencies by 2000 (Walker 2001: 6). For 
this report, we identihed more than 140 civilian oversight agencies, with almost 
all large cities having some sort of civilian oversight mechanism in place. More 
critically, roughly 55 percent of the oversight agencies that provided data for this report indicated that they had been in ex¬ 
istence for 16 or more years. These patterns, when taken as a whole, seem to indicate a held that is approaching maturity. 
Large jurisdictions that do not have oversight now stand as outliers, rather than representing the norm. 

Oversight Typically Develops Through Local Grises 

The history of oversight in the U.S. demonstrates that the early models of oversight emerged in response to local crises re¬ 
lating to conhict between the police and local communities of color, often catalyzed by controversial officer-involved shootings, 
in-custody deaths or concerns about racially disparate policing (Walker 2001). Few historical examples of oversight exist of 
proactive implementation. The data provided by civilian oversight agencies for this report supports this view. Only 11 percent 
of the agencies reported proactive implementation of their oversight system. While not common, this form of development 
suggests that it is possible for local jurisdictions to implement civilian oversight in a proactive versus reactive manner. 

Oversight Is a Complex Organizational Field 

In looking at the historical evolution of oversight, it is clear that earlier forms of oversight tended to be organized around the 
review of completed internal affairs investigations. The held has evolved considerably, however. Review-focused agencies 


These patterns, when taken as a whole, seem 
to indicate a field that is approaching 
maturity. Large jurisdictions that do not have 
oversight now stand as outliers, rather than 
representing the norm. 
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were the most common among the respondents to provide data (40 percent), but auditor/monitor and investigation-focused 
agencies are now common in the held (25 percent and 35 percent, respectively). 

This means that local jurisdictions have a much broader range of options when considering the characteristics of the oversight 
mechanism they would like to implement. 

In the data shared by the civilian oversight agencies, it was clear that each of the models identihed tended to have common orga¬ 
nizational characteristics and forms of authority. 

• Review-focused agencies, on the whole, were more likely to report that they had jurisdiction in relation to the review of 
completed internal affairs investigations. 

• Investigative agencies were more likely to report that they had authority to decide how to handle citizen complaints and 
conduct independent investigations. 

• Auditor/monitor agencies were more likely to report having strong authority in relation to reviewing, auditing and mon¬ 
itoring investigations, as well as authority to analyze patterns and make recommendations across a range of police/sheriff 
data. 

Even though the oversight models tended to have clear organizational foci, there was substantial crossover among models. 
For example, a majority of auditor/monitor agencies (61 percent) reported the ability to conduct independent investigations.^® 
Roughly 43 percent of review-focused agencies and 50 percent of investigative agencies reported having the authority to 
audit complaint investigations by the police. Taken as a whole, this indicates that jurisdictions are mixing-and-matching forms 
of oversight authority and jurisdiction to meet the needs in the local environment (McDevitt et al. 2005: 11). As such, regard¬ 
less of whether a local jurisdiction has a review board or auditor/monitor, it is clear that just about any type of oversight 
model can include hybridized characteristics. 

There is Clear Variation in the Budgets Among Oversight Types 

In general, the data provided by the civilian oversight agencies demonstrated that some models of oversight have larger 
budgets than others. Full investigative agencies tended to report the largest budgets, followed by auditor/monitor and review 
agencies. Similar patterns were evident in terms of staffing. Investigative agencies reported having the largest number of 
full-time, paid staff while review-focused agencies were the least likely to report having full-time paid staff. This is likely due 
in part to variation in the size of jurisdictions and variation in the mandate of the different models. Investigative and auditor/ 
monitor agencies were much more hkely to be found in jurisdictions with large populations and big pohce agencies. 

Even so, jurisdictions need to carefully evaluate the trade-offs between different types of oversight agencies. Review-focused 
agencies are the least expensive agencies to operate. Yet, the pattern in data provided by the civilian oversight agencies indi¬ 
cate that review-focused agencies also tend to have the most limited range of organization¬ 
al authority. Focal jurisdictions will need to carefully match their goals for oversight to the 
resources they have available when considering the type of oversight to implement. 

Some Models of Oversight May be More EflFective at Getting 
Recommendations Implemented 

Almost all of the oversight agencies reported that police executives listened carefully to the 
recommendations made by oversight staff (78 percent). However, auditor/monitor agencies 
were much more likely to report that police or sheriffs agencies implemented their recom- 


29 Olson and Attard (2016), in a review of oversight models in California, found that auditors were often granted the legal authority to conduct individual investigations, but seldom 

exercised that authority. 


Even though a large majority 
of all oversight agencies report¬ 
ed that police executives listened 
to their recommendations, 
auditor/monitor agencies were 
more likely to report that law 
enforcement frequently imple¬ 
mented their recommendations. 
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mendations very frequently or frequently (72 percent) as compared to investigative (42 percent) and review agencies (34 
percent). It is critical to note that some caution is warranted when it comes to drawing conclusions from this hnding. These 
are self-report data and thus only measure the perceptions of the oversight executives who provided input. 

Part of the difference, however, may relate to the type of work different agencies tend to perform. Auditor/monitor agencies 
are more likely to have specialized staff who focus on analyzing patterns in complaints and other poKce data and they are 
more likely to issue special topics reports as a result of that work (which Harris (2012) describes as a “holistic’ approach). It 
is a possibility that using data-driven evaluation methods increases the likelihood that police departments will implement 
recommendations made by oversight. More systematic comparative research is needed; however, before strong inferences 
can be drawn on this issue. 

Oversight Gan Serve as a Conduit of Positive Information about Police Department Performance 

Even though much of the public reporting on police accountability and oversight tends to focus on instances of wrong-doing 
or misconduct on the part of individual ofhcers, police oversight agencies can also serve as an important conduit of infor¬ 
mation to the public about the positive forms of police-community interaction. A number of jurisdictions actively seek to 
use their public reporting frameworks to highlight circumstances where police officers have engaged in outstanding public 
service. For example, Albuquerque’s Civilian Police Oversight Agency (2015) and Denver’s Office of the Independent Monitor 
(2015) developed sections of their annual reports to highlight commendations and other notable positive forms of police-com¬ 
munity interaction. 
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Important Considerations when Implementing or 
Reforming a Loeal Oversight Program 

Over the past 30 years, local experimentation with different types of oversight models, to include the hybridization of these 
different models, has result in a complex heterogeneous organizational held. While the data provided by 97 civilian oversight 
agencies included in this report explores organizational variation across different oversight agencies, it cannot answer the 
fundamental question of: 

• Which forms of oversight are the most effective? 

• Under what circumstances should a jurisdiction implement a review-focused model of oversight as opposed to an 
investigative or auditor/monitor-focused model? 

Even though the question of what type of model constitutes a “best” form of oversight remains unanswered, that does 
not mean that nothing can be learned from these patterns. In fact, the growing hybridization of police oversight and the 
blurring of the boundaries between different models of oversight by itself carries with it an important lesson for local 
jurisdictions that are exploring whether to implement oversight or considering revising their current oversight framework. 

Jurisdictions Should Focus on “Best-Fit” Rather Than “Best Practices” When Considering How 
to Structure Civilian Oversight 

One key lesson that can be learned from the history of oversight in the U.S. is that there is 
not necessarily any “best practice” in the creation of civilian oversight of a law enforcement 
program, but rather what a jurisdiction should be looking for is a “best htting” model of 
oversight. Every jurisdiction has its own social, cultural and political issues and every police 
agency has its own unique organizational, history, traditions and sub-cultural characteristics. 
While some police agencies may be prohcient at holding their officers to account with respect 
to certain types of conduct, other police agencies may struggle. Some large jurisdictions have 
ample hnancial resources to implement highly professionalized, organizationally complex 
forms of oversight while smaller jurisdictions may have far fewer resources with which to 
implement and sustain police oversight. 

Given these differences between cities and counties in the U.S., it is likely that no single model 
of oversight is going to work for all jurisdictions. As a result, the best form of oversight for individual jurisdictions simply depends 
on the circumstances faced by the jurisdiction that is either creating or updating its oversight processes. This is why it is import¬ 
ant to review the strengths and challenges of a police department before selecting an oversight model. 

Oversight Should Employ the Least Force Necessary to Accomplish its Goals 

Even though law enforcement resistance to the concept of police oversight has diminished over time, it can still be argued that 
“the least intrusive means of oversight” (Bobb 2003) necessary to achieve police accountability is the best means of approaching 
the oversight function in the long term. Just as we expect the police to use only that amount of force that is proportionate, nec¬ 
essary and reasonable to accomplish their task, so it can be argued that jurisdictions creating or reforming an oversight function 
should similarly accomplish the feat of ensuring police accountability (Bobb 2003; 165). In other words, a jurisdiction seeking 
to create or update an oversight function should choose the least intrusive model of oversight necessary to accomplish the task. 
If the model chosen does not accomplish that objective, a more aggressive form of oversight would then be required. As such, 
it is impossible to suggest that any one model of oversight is better than another. Each jurisdiction must evaluate its own police 


“Evidence that that any one 
civilian oversight approach or 
mechanism is more effective than 
another does not yet exist, 
although the role and authority of 
a civilian oversightfiinction often 
grows over time to meet emerging 
community needs and expecta¬ 
tions. ” (Anderson et al. 2015:3) 
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agency, including its culture, leadership, capacity to police itself before choosing the most appropriate form of oversight. 

A Number of Resources are Available to Jurisdictions Considering Implementing Oversight or 
Reforming their Current Oversight Framework 

One of the key challenges for local jurisdictions considering whether to implement oversight is to hnd examples of jurisdic¬ 
tions that have successfully implemented and sustained effective oversight agencies. It can also be difhcult and resource in¬ 
tensive for local jurisdictions to collect examples of legal language, organizational procedures and other “nuts-and-bolts 
documents that they can use as models after they decide to implement oversight. Several relatively recent reports have sought 
to overcome these problems by providing detailed cases studies of existing oversight agencies (PARC 2005; McDevitt et al. 
2005; Noe 2013; Olson 2016; PARC 2016). These reports contain key details about oversight agency powers, organization¬ 
al, structure, funding and staffing and should be consulted by local jurisdictions considering oversight or interested in reform¬ 
ing their local oversight agency. 

Recommendations for Further Research 

The following recommendations, which emerged from the review of 
the literature and organizational data provided by civilian oversight 
agencies, are intended to identify ways to expand what is currently 
known about civilian oversight of law enforcement. 

1. More systematic, comparative research needs to be conducted on 
the effectiveness of civilian oversight of law enforcement. While 
some researchers are working to extend what we know about oversight 
(see Walker and Archbold 2014; Prenzler and den Heyer 2016; Alpert 
et al. 2016), there are still tremendous gaps in what we know about 
the issue of effectiveness in civilian oversight. Moreover, we stiU know 
little about the relative effectiveness of different models, strategies 
and approaches. While the data provided by nearly 100 civilian 
oversight agencies helps to identify and examine broad differences 
among models of oversight, it does not allow for the measurement of differences in the impact that different types of 
oversight agencies have on police accountability, police effectiveness or patterns in police misconduct. It is clear that 
more systematic, empirical research is needed before we will be able to draw strong conclusions about the organization¬ 
al factors that determine effectiveness in citizen oversight. 

2. Additional research is needed on the impact that citizen oversight has on public trust in the police and local government. 
Civilian oversight often emerges as a result of conflict between the police and communities of color and is typically 
catalyzed by one or more local police scandals (e.g, racially biased policing, officer-involved shootings) (Walker, 2001, 
p. 55). Moreover, oversight is often implemented with the goal of improving community trust and conhdence in the 
police and local government. While some scholars have begun to study the relationship between civilian oversight and 
public opinion (Tarling and Dowds 1997; Worden and McLean 2010; Prenzler 2016; De Angelis and Wolf 2016), we 
StiU know little about the impact that the implementation of civUian oversight has on public opinion. We also know little 
about the extent to which the presence of civilian oversight can repair the damage to public conhdence caused by in¬ 
stances of police misconduct. More research is needed on the impact of civilian oversight on public trust in policing 
and local government. 

3. Future comparative evaluations of effectiveness need to be based on primary data collected by outside researchers. Prenzler 
and Lewis (2005) and Ferdik et al. (2013) have conducted some of their most important work by reviewing publicly-avaU- 


To help local jurisdictions gain access to examples of 
oversight policies, legal language and key organiza¬ 
tional documents, the National Association for 
Civilian Oversight of Law Enforcement (NACOLE) 
has created a companion website to this report that 
includes up-to-date profiles for model police oversight 
agencies. This website toolkit includes examples of 
ordinance/charter language, oversight policies and 
procedures, annual reports, special topics reports, 
complaint forms, outreach brochures and examples of 
other documents that can serve as examples for new 
oversight agencies. This website toolkit can be 
accessed by visiting: www.nacole.org/agency profiles . 
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able agency reports and documents. However, the lack of standardization in reporting and measurement among agencies 
makes it difficult to compare across jurisdictions. Future research should adopt mixed-methods methodological approach¬ 
es that combine the use of quantitative outcome measurement with qualitative interviews with community members and 
officers to generate forms of data that allow for comparison across jurisdictions (Ferdik et al 2013; 18). 

4. More support for independent research on oversight will be necessary before key questions relating to the effectiveness 
of oversight can be answered. Federal and state agencies that support applied research in law enforcement should con¬ 
sider developing new and sustained funding streams for research into civilian oversight of police (Buchner 2015). The 
same applies to private funding sources that support social science research and study. 

5. More work on standards for oversight is needed. Professional oversight associations, like NACOLE, should continue to 
work coUaboratively with researchers and other oversight stakeholders to develop standards and guidelines for evaluat¬ 
ing oversight. 

6. Local jurisdictions should consider participating in the “open data” movement. Over the last several years, a number 
of jurisdictions have begun to make some of their non-identifying police-related data available to the public, including 
data related to complaints against officers, critical incidents and other data relating to police operations and activity. 
The goal of this has been to build public trust in the police and local government by increasing the transparency of 
police operations and local government. For example, the Indianapolis Metropolitan Police Department and Citizen’s 
Police Complaint Office collaborated with a local non-profit. Code for America, to create a web portal through which 
members of the public can download datasets relating to officer uses of force, officer-involved shootings and complaints 
filed by members of the public ( https://www.projectcomport.org ). 

7. Local jurisdictions should be willing to evaluate the effectiveness of their local oversight agencies proactively and inde¬ 
pendently. In conducting those evaluations, jurisdictions need to have realistic expectations when it comes to the impact 
civilian oversight may have on local police accountability. Oversight is—and should be—only one component of a ju¬ 
risdiction’s police accountability framework. As such, local jurisdictions should seek to evaluate effectiveness based on 
the factors over which local oversight agencies have control. As the lACP noted: “Citizen review is but one tool among 
many that can be used to promote and ensure accountability. It is neither a cure-all not likely to promote desired results 
unless accompanied by a full package of accountability-building strategies” (lACP 2000; 2). 
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Appendix A. 

Research Methods 

Since much of the recent research on civilian oversight has been based on either case studies or anecdotal accounts of indi¬ 
vidual agency practices, there are substantial gaps in what is known about the state of civilian oversight in the 21 st century. 
To develop a better understanding of the state of civilian oversight in the US., the research team administered an electron¬ 
ic organizational survey to 97 civilian oversight agencies in the US. The overall goal of the survey was to capture and describe 
variation in the organizational structure, legal authority, organizational practices and financial resources for contemporary 
oversight in the US. Several of the attitudinal questions were adapted from early survey research on oversight (Walker and 
Bumphus 1991). The survey instrument contained 86 item groups that were designed to measure organizational and attitu¬ 
dinal information in relation to 12 key areas: 

• The history of the oversight agency. 

• The legal authority/organizational powers granted to the agency. 

• Agency goals organizational structure. 

• Budget. 

• Staffing. 

• Presence and structure of a board or community members. 

• Role in the complaint intake and investigation process. 

• Community outreach practices. 

• Perceived support from process stakeholders. 

• Data collection and analysis practices. 

• Key successes or obstacles that the agency has experienced in the previous three years. 
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Prior to the distribution of the surveys, the authors hrst sought to identify local and state-level civilian oversight agencies cur¬ 
rently operating in the U.S. For the purposes of this portion of the project, civilian oversight agencies were dehned as govern¬ 
mental bodies that involve non-police community members in the review, auditing, monitoring or investigation of allegations 
of misconduct hied against law enforcement ofhcers (see Walker 2001). Generating a list of oversight agencies proved to be a 
substantial challenge. There are currently no governmental or regulatory bodies that precisely track or maintain current lists of 
all oversight agencies in the U.S. The project team used several methods to identify civilian oversight agencies. 

1. Reviewed organizational membership lists for NACOLE; 

2. Reviewed the academic literature and all available policy evaluation reports for references to oversight agencies; 

3. Used U.S. Census data to identify all U.S. cities with greater than 100,000 residents (N=295) (United States Census Bureau 
2014). Once the list was developed, the research team conducted a series of key word searches of the Internet using Google 
and a combination of the city names and terms relating to civilian oversight. Reviewed the resulting websites to determine 
whether a civilian oversight agency was in operation and to collect contact information (when available). 

4. Conducted key word searches of newspaper articles and other print media sources using Google News and LexisNexis 
Academic. 

5. Consulted subject matter experts knowledgeable about civilian oversight (e.g, individuals with experience working in 
the area of civilian oversight). 

Once an initial list of oversight agencies was developed, the team collected and stored all available substantive website content, 
including contact information, electronically. For those identihed agencies where contact information for the oversight exec¬ 
utive could not be determined, used emails and phone interviews to; 

1. Determine whether the oversight agency was in operation; and, 

2. Collect the contact email information for the oversight executive (or his or her designee). 

Altogether, the team identihed 144 oversight agencies using these methods. Twenty-one of those jurisdicrions either refused 
to provide electronic contact information for the oversight agency or failed to respond to multiple email and phone call re¬ 
quests for contact information. The research team did not send surveys to four agencies because the contractor operating 
their oversight body performs oversight duties in another city (each contractor was surveyed only once). This left the research 
team with viable contact information for 119 civilian oversight agencies. 

Once the team identihed the agencies, the team developed and administered the survey using Qualtrics™ survey research 
software. The University of Idaho Institutional Review Board reviewed and approved the survey research design, protocol 
and protections for human subjects. Each oversight agency received up to hve separate contacts. Each agency received an 
initial notihcation email from the president of NACOLE that explained the nature, goals and likely outcome of the survey. 
The oversight agencies then received a solicitation email with a link to the survey three days later. The team delivered three 
additional solicitations via email to non-responding agencies. After the fourth solicitation, the research team and NACOLE 
board members and staff placed a hnal round of follow-up phone calls to the non-responding agencies with a request for 
their participation. In total, 97 of the 119 oversight agencies that received a survey solicitation submitted an electronic survey, 
resulting in a response rate of 81.5 percent. Once the team received the survey data, they extracted the data from Qualtrics™ 
and analyzed it in SPSS. 

When reviewing the results of the survey, it is important to recognize several important limitations. First, even though the 
team identihed 144 oversight agencies, the research team was unable to distribute surveys to 25 of those agencies. In addition. 
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another 22 agencies received, but did not submit, a survey. This indicates that some caution should be used when attempting 
to draw strong inferences about the larger population of civilian oversight agencies in the U.S. Second, the team distributed 
the electronic survey to only one representative from each agency. It is not clear whether the attitudes reported by the agency 
representatives reflect the attitudes of other employees or volunteers working for those jurisdictions. 

Appendix B. 

Statistical Results 

Table Bl; Percentage of Organizational Goals Rated as “Critical” or “Important” 


by Oversight Agencies 



Auditor/ 

monitor 

Investigative 

Review 

All Agencies 

Improve public trust in the local police/sheriff department 

100 % 

97 % 

97 % 

98 % 

Ensure that complaint processes are accessible to aU members of 
the community 

87 % 

100 % 

91 % 

93 % 

Ensure thorough investigations of complaints 

87 % 

97 % 

82 % 

89 % 

Increase transparency of the complaint investigation system 

91 % 

85 % 

85 % 

87 % 

Deter police misconduct 

91 % 

91 % 

74 % 

85 % 

Ensure that individuals who experience police misconduct receive 
justice 

74 % 

85 % 

71 % 

77 % 

Make recommendations to the police about how to improve the 
investigations process 

100 % 

62 % 

71 % 

75 % 

Improve complainant satisfaction with the complaint process 

70 % 

91 % 

65 % 

76 % 

Look for patterns in complaints to identify the underlying causes of 
police misconduct 

87 % 

85 % 

52 % 

73 % 

Analyze patterns in police data to make recommendations for 
organizational improvement 

87 % 

76 % 

38 % 

64 % 

Issue public reports that describe patterns in complaints 

73 % 

68 % 

44 % 

60 % 

Ensure that officers who engage in misconduct are punished 

61 % 

65 % 

44 % 

56 % 

Improve police officer satisfaction with the complaint process 

52 % 

53 % 

53 % 

53 % 

Audit internal investigations for investigatory compliance 

74 % 

47 % 

32 % 

48 % 

Use alternative dispute resolution techniques to resolve complaints 
(e.g, mediation) 

52 % 

56 % 

35 % 

47 % 

Use performance audits to strengthen police practices, policies or 
training 

57 % 

29 % 

30 % 

37 % 

Sustain a proper percentage of complaints 

18 % 

26 % 

35 % 

28 % 

Number of Agencies 

23 

34 

34 

91 
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Table Bl: Percentage of Organizational Goals Rated as “Critical” or “Important” 
by Oversight Agencies 

Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the 
number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of 
agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may 
vary very slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 


Table B2; What Local Circumstances Triggered the Creation of your Oversight Agency? 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Officer-Involved Shooting/Other Excessive Force 

50% 

56% 

44% 

49% 

Racially Disparate Policing (e.g, racial profiling) 

25% 

38% 

23% 

29% 

Proactive/Preventive Effort 

8% 

0% 

23% 

11% 

In-Custody Death 

13% 

6% 

10% 

9% 

Police Corruption Scandal 

13% 

12% 

5% 

9% 

Change by Legislative Body 

0% 

12% 

13% 

9% 

Sexual Misconduct 

8% 

0% 

5% 

4% 

Mishandled internal Affairs Investigations 

4% 

0% 

3% 

2% 

Dissatisfaction with Previous Oversight Agency 

8% 

0% 

0% 

2% 

Other Issue 

21% 

9% 

13% 

13% 

Number of Agencies 

24 

34 

39 

97 

Table Notes: Agencies could select multiple responses, so percentages will sum 
excludes missing values from the calculation. 

to more than 100%. The table reports the valid percent and 

Table B3: What type of Law Enforcement Organization(s) do you Oversee? 


Auditor/ 

monitor 

Investigative 

Review 


All 

Agencies 

Municipal Police Department 

67% 

85% 

90% 

82% 

County Sheriff 

25% 

12% 

13% 

15% 

Campus Law Enforcement 

4% 

3% 

8% 

5% 

Special Police Agency 

4% 

6% 

3% 

4% 

Probation 

4% 

3% 

0% 

2% 

Number of Agencies 

24 

34 

39 

97 

Table Notes: Agencies could select multiple responses, so percentages will sum 

to more than 100%. The table reports the valid percent and excludes 


missing values from the calculation. 
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Table B4; Approximately how Many Sworn Oflficers Work for the Agencies you Oversee? 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

500 or Fewer 

29% 

34% 

64% 

45% 

501 to 1,000 

21% 

22% 

21% 

21% 

1,001 to 2,000 

29% 

16% 

10% 

17% 

2,001 to 3,000 

4% 

13% 

0% 

5% 

3,001 to 4,000 

0% 

9% 

3% 

4% 

4,001 or more 

17% 

6% 

3% 

7% 

Number of Agencies 

24 

32 

39 

95 

Table Notes: The table reports the valid percent and excludes missing values from the calculation. 


Table B5: Who Leads your Oversight Agency? 


Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Full-Time Paid Executive 

78% 

71% 

39% 

61% 

Part-Time Paid Executive 

17% 

6% 

6% 

9% 

Multi-Member Board 

4% 

24% 

52% 

29% 

Other 

0% 

0% 

3% 

1% 

Number of Agencies 

23 

34 

33 

90 

Table Notes: The table reports the valid percent and excludes missing values from the calculation. 
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Table B6; What is the Source of your Agency’s Authority? 



Auditor/ 

Investigative 

monitor 

Review 

All 

Agencies 

County/Municipal Ordinance 

25% 

41% 

36% 

35% 

City/County Charter 

29% 

34% 

38% 

35% 

Both Ordinance/Charter 

8% 

0% 

0% 

2% 

State Statute 

8% 

13% 

13% 

12% 

Contract 

17% 

3% 

0% 

5% 

Executive Order 

4% 

6% 

0% 

3% 

Law Enforcement Policy 

4% 

3% 

3% 

3% 

City/County Policy 

4% 

0% 

3% 

2% 

Legal Agreement 

0% 

0% 

5% 

2% 

Other 

0% 

0% 

3% 

1% 

Number of Agencies 

24 

32 

39 

95 

Table Notes: Agencies could select multiple responses, 
missing values from the calculation. 

so percentages will sum to more than 100%. The table reports the valid percent and excludes 


Table B7: What Type of Job Protection Does the Head of your Agency have? 


Auditor/ 

Investigative 

monitor 

Review 

All 

Agencies 

At Will 

73% 

67% 

48% 

78% 

Contract 

18% 

11% 

17% 

15% 

Term of Office 

9% 

15% 

22% 

15% 

Removed by Legislature/City Council 

9% 

7% 

9% 

8% 

Civil Service 

0% 

7% 

4% 

4% 

Other 

18% 

0% 

17% 

21% 

Number of Agencies 

22 

27 

23 

72 


Table Notes: This question was only administered to oversight agencies who reported that they were led by a full or part-time executive. Agencies 
could select multiple responses, so percentages will sum to more than 100%. The table reports the valid percent and excludes missing values from 
the calculation. 
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Table B8; Are Former Police Oflficers Allowed to Serve as your Agency Executive/Director? 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Yes 

52% 

63% 

32% 

50% 

Under some circumstances 

33% 

33% 

41% 

36% 

No 

14% 

4% 

27% 

14% 

Number of Agencies 

21 

27 

22 

70 


Table Notes: This question was only administered to oversight agencies who reported that they were led by a full or part-time executive. The table 
reports the valid percent and excludes missing values from the calculation. 


Table B9: Percentage of Agencies Reporting that they “Always/Sometimes” have Jurisdiction in Specif¬ 
ic Areas of the Personnel Investigations Process 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Citizen-Initiated Complaints 

92% 

100% 

92% 

95% 

Officer-Initiated Complaints 

83% 

49% 

50% 

58% 

Internal/ Supervisor Complaints 

87% 

39% 

49% 

55% 

Officer-Involved Shootings 

86% 

76% 

71% 

77% 

Other Serious Force 

86% 

85% 

77% 

82% 

In-Custody Deaths 

86% 

69% 

66% 

72% 

High Speed Pursuits 

86% 

56% 

63% 

66% 

Complaints against Non-Sworn Employees 

67% 

36% 

20% 

38% 

Criminal Investigations 

61% 

16% 

27% 

32% 

Tort Claims 

52% 

38% 

23% 

36% 

Number of Agencies 

24 

33 

38 

95 


Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the 
number of agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of 
agencies that had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may 
vary very slightly from tlie total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 
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Table BIO; Percent of Agencies Responding with a “Yes” or “Sometimes” that they Have Specific 
Forms of Authority in Relation to Citizen Complaints 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Classify the nature of a complaint 

57% 

94% 

39% 

63% 

Conduct an initial intake investigation on complaints 

52% 

94% 

30% 

58% 

Decide how a complaint will be handled (e.g., investigated, administra¬ 
tively closed, mediated) 

48% 

97% 

27% 

57% 

Review police complaint investigations (e.g., for thoroughness, complete¬ 
ness, accuracy) 

96% 

67% 

81% 

80% 

Send complaint investigations back to the police for further investigation 

78% 

53% 

65% 

64% 

Audit police complaint investigations for compliance with investigative 
standards 

92% 

50% 

43% 

58% 

Monitor open police complaint investigations 

92% 

52% 

38% 

57% 

Conduct investigations that are independent of the police 

61% 

100% 

32% 

63% 

Recommend investigation hndings to the police 

70% 

84% 

57% 

70% 

Issue hndings on investigations 

61% 

88% 

54% 

67% 

Recommend discipline to the police 

61% 

53% 

30% 

46% 

Impose Discipline 

4% 

10% 

3% 

6% 

Attend police disciplinary hearings 

46% 

39% 

14% 

31% 

Attend police use of force hearings 

52% 

33% 

16% 

31% 

Assist in the operation of a police early intervention system 

52% 

33% 

16% 

31% 

Number of Agencies 

23 

33 

37 

93 


Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the 
number of agencies that responded to each specific item. The total number of agencies reported at tire bottom of the table identifies tire number of 
agencies tliat had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may 
vary very slightly from tire total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 
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Table Bll; Percentage of Agencies Reporting that they “Always/Sometimes” Have the Following Types 
of Authority in Relation to Police Records? 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Access to CLOSED internal affairs police files 

96% 

59% 

57% 

68% 

Access to ALL internal affairs police files 

88% 

52% 

38% 

55% 

Access to internal affairs electronic databases 

75% 

48% 

30% 

48% 

Access to all electronic police databases 

58% 

39% 

19% 

36% 

Access to Body Worn/In-Car Video 

83% 

78% 

65% 

74% 

Right to subpoena witnesses 

30% 

56% 

32% 

40% 

Right to subpoena records 

26% 

59% 

35% 

41% 

Number of Agencies 

24 

33 

37 

94 


Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the 
number of agencies that responded to each specific item. The total number of agencies reported at tlie bottom of the table identifies tire number of 
agencies tliat had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may 
vary very slightly from tire total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 


Table B12; Percent of Oversight Agencies Indicating “Strongly Agee/Agree” that Police Stakeholders 
Listened Carefully to Recommendations Made by Oversight Staff 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Police executives listen carefully to recommendations made by oversight 
staff 

83% 

76% 

78% 

78% 

Number of Agencies 

18 

33 

32 

83 


Table Notes: The table reports the valid percent and excludes missing values from the calculation. 


Table B13; Percent of Oversight Agencies Indicating that Police Stakeholders “Very Frequently/Fre¬ 
quently” Implemented their Recommendations 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

How often does the police/sheriff implement recommendations made 
by your oversight agency? 

72% 

42% 

34% 

46% 

Number of Agencies 

18 

33 

32 

83 


Table Notes: The table reports the valid percent and excludes mis.sing values from the calculation. 
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Table B14: Percentage of Oversight Agencies Indicating that their Enabling Statute/Ordinance Explic¬ 


itly Includes a Requirement that Law Enforcement Employees Cooperate with their Agency 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

That police officers/command staff cooperate with your oversight 
agency? 

52% 

73% 

40% 

55% 

That police officers/command staff cooperate with your oversight 
agency work as a condition of their employment? 

26% 

47% 

20% 

31% 

Number of Agencies 

23 

33 

35 

91 


Table Notes: The table reports the valid percent and excludes missing values from the calculation. 


Table B15: Percentage of Oversight Agencies who Indicated “Very Strong/Strong” Support from Key 
Process Stakeholders 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Police Executive 

83% 

87% 

93% 

88% 

Police Command Staff 

78% 

81% 

93% 

85% 

Internal Affairs Command Staff 

88% 

73% 

92% 

84% 

Internal Affairs Investigators 

82% 

73% 

83% 

79% 

Police Union 

35% 

21% 

55% 

35% 

City Attorney/County Counsel 

47% 

74% 

75% 

67% 

Mayor 

75% 

81% 

82% 

80% 

City Council 

92% 

82% 

88% 

86% 

Civil Rights Community 

82% 

71% 

65% 

72% 

Number of Agencies 

18 

31 

29 

78 


Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the 
number of agencies that responded to each specific item. The total number of agencies reported at tire bottom of the table identifies tire number of 
agencies tliat had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may 
vary slightly from the total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 
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Table B16: Yearly Operating Budget for Responding Oversight Agencies 


Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

No budget 

12% 

3% 

59% 

26% 

$1 -$10,000 

0% 

3% 

10% 

5% 

$10,001 -$100,000 

12% 

7% 

10% 

9% 

$100,001 -$250,000 

6% 

13% 

7% 

9% 

$250,001 -$500,000 

24% 

20% 

10% 

17% 

$500,001 -$1,000,000 

18% 

17% 

0% 

11% 

More than 1,000,000 

29% 

37% 

3% 

22% 

Number of Agencies 

17 

30 

29 

76 

Table Notes: The table reports the valid percent and excludes missing values from the calculation. 


Table B17; Number of Full Time, 

Paid Staff by Type of Oversight Agency 





Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

0 Staff 

26% 

12% 

61% 

33% 

1-5 Staff 

43% 

38% 

27% 

36% 

6-10 Staff 

9% 

26% 

3% 

13% 

11-15 Staff 

9% 

3% 

3% 

4% 

16-20 Staff 

4% 

6% 

6% 

6% 

21 or more Staff 

9% 

15% 

0% 

8% 

Number of Agencies 

23 

34 

33 

90 


Table Notes: The table reports the valid percent and excludes missing values from the calculation. 


Table B18: Does your Agency Publish Public Reports? 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Yes 

80% 

85% 

69% 

78% 

No 

20% 

15% 

31% 

22% 

Number of Agencies 

20 

33 

32 

85 


Table Notes: The table reports the valid percent and excludes missing values from the calculation. 
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Table B19; Types of Issues Analyzed “Very Frequently/Frequently” by Oversight Agencies 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Total number of complaints filed 

67% 

76% 

42% 

61% 

Types of allegations contained in complaints 

71% 

76% 

39% 

61% 

Findings on allegations 

71% 

67% 

36% 

56% 

Timeliness data 

48% 

58% 

33% 

46% 

Other uses of force 

57% 

48% 

27% 

43% 

Officer-involved shootings 

62% 

33% 

30% 

39% 

Discipline imposed 

52% 

36% 

18% 

33% 

In-custody death 

57% 

30% 

18% 

32% 

Traffic stop data 

29% 

12% 

21% 

20% 

Demographic information of individuals subject to stops, searches or 

arrest 

29% 

21% 

12% 

20% 

Vehicle pursuits 

29% 

18% 

15% 

20% 

K-9 bites 

24% 

15% 

21% 

20% 

Tort claims hied against officers 

19% 

15% 

18% 

17% 

Agency Does Not Analyze Statistical Patterns 

10% 

18% 

33% 

22% 

Number of Agencies 

21 

33 

33 

87 


Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the 
number of agencies that responded to each specific item. The total number of agencies reported at tire bottom of the table identifies tire number of 
agencies tliat had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may 
vary very slightly from tire total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 
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Table B20: Did Representatives from your Agency Conduct Outreach Activities with any the Following 
Groups in the Last Year? 



Auditor/ 

monitor 

Investigative 

Review 

All 

Agencies 

Community leaders 

83% 

88% 

63% 

77% 

Neighborhood associations 

72% 

76% 

59% 

69% 

Policy-makers (e.g., city council members) 

78% 

70% 

53% 

65% 

Police command staff 

72% 

64% 

47% 

59% 

Police officers 

67% 

55% 

50% 

59% 

Schools 

61% 

45% 

38% 

46% 

Religious organizations 

44% 

39% 

38% 

40% 

Youth group 

56% 

42% 

25% 

39% 

Mental health community 

61% 

27% 

19% 

31% 

Police unions 

61% 

33% 

9% 

30% 

At-risk youth 

50% 

27% 

16% 

28% 

Homeless community 

50% 

18% 

16% 

24% 

Health care providers 

17% 

15% 

6% 

12% 

Inmates 

17% 

12% 

6% 

11% 

Others 

6% 

6% 

31% 

16% 

Number of Agencies 

18 

33 

32 

83 


Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the 
number of agencies that responded to each specific item. The total number of agencies reported at tlie bottom of the table identifies tire number of 
agencies tliat had valid responses to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may 
vary very slightly from tire total reported at the bottom of the table. However, any differences are reflected in the reported percentages. 
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Table B21; Most Common Outreach Activities Reported by Oversight Agencies in the Previous Year 



Auditor/monitor 

Investigative 

Review 

All Agencies 

Operated an agency website 

72% 

88% 

47% 

69% 

Tabled at community fairs 

56% 

52% 

41% 

48% 

Provided complaint forms at multiple locations 

56% 

58% 

28% 

46% 

Held community forums 

50% 

48% 

38% 

45% 

Operated an agency Facebook page 

28% 

45% 

25% 

34% 

Mailed brochures/business cards to community groups 

28% 

30% 

31% 

30% 

Held press conferences 

33% 

30% 

13% 

24% 

Mailed brochures/business cards to neighborhood centers 

28% 

27% 

16% 

23% 

Operated an agency Twitter account 

33% 

24% 

9% 

20% 

Posted signs in police stations 

22% 

24% 

9% 

18% 

Televised hearings 

11% 

24% 

13% 

17% 

Other 

11% 

9% 

28% 

17% 

Number of Agencies 

18 

33 

32 

83 


Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of 
agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses 
to one of more items listed in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the 
bottom of the table. However, any differences are reflected in the reported percentages. 


Table B22: Percentage of Agencies Indicated that they have Translated Outreach Documents into 
Languages Other Than English? 



Auditor/monitor 

Investigative 

Review 

All Agencies 

Complaint forms 

53% 

59% 

25% 

44% 

Agency brochures 

59% 

56% 

34% 

48% 

Agency web page 

24% 

22% 

3% 

15% 

Agency reports 

12% 

6% 

3% 

6% 

Other 

6% 

9% 

0% 

5% 

Number of Agencies 

17 

32 

32 

81 


Table Notes: Table reports the valid percent for each item and excludes missing values from the calculation. Each valid percent was calculated using the number of 
agencies that responded to each specific item. The total number of agencies reported at the bottom of the table identifies the number of agencies that had valid responses 
to one of more items fisted in the table. As a result, the number of agencies responding to each item in the table may vary very slightly from the total reported at the 
bottom of the table. However, any differences are reflected in the reported percentages. 


Table B23: Does your Agency have a Board Composed of Community Members? 



Auditor/monitor 

Investigative 

Review 

All Agencies 

Yes 

65% 

85% 

100% 

86% 

No 

35% 

15% 

0% 

14% 

Number of Agencies 

23 

34 

37 

94 


Table Notes: The table reports the valid percent and excludes missing values from the calculation. 
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